-- Logs begin at Mon 2022-09-19 15:33:37 UTC, end at Mon 2022-09-19 17:03:23 UTC. -- Sep 19 16:04:41.602309 dv-cinder-858370-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 19 16:04:41.614882 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 16:04:41 2022] *** Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: nodename: dv-cinder-858370-2-iscsi Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: machine: x86_64 Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: clock source: unix Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: pcre jit disabled Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: detected number of CPU cores: 4 Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: current working directory: / Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: detected binary path: /usr/bin/uwsgi-core Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: your processes number limit is 47680 Sep 19 16:04:41.618063 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: your memory page size is 4096 bytes Sep 19 16:04:41.619230 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: detected max file descriptor number: 2048 Sep 19 16:04:41.619230 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: lock engine: pthread robust mutexes Sep 19 16:04:41.619230 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: thunder lock: enabled Sep 19 16:04:41.619230 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 19 16:04:41.619230 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 16:04:41.658574 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: Python main interpreter initialized at 0x55815a4819c0 Sep 19 16:04:41.658574 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: python threads support enabled Sep 19 16:04:41.658574 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: your server socket listen backlog is limited to 100 connections Sep 19 16:04:41.658574 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: your mercy for graceful operations on workers is 90 seconds Sep 19 16:04:41.659190 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 16:04:41.659190 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: *** Operational MODE: preforking *** Sep 19 16:04:41.659423 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: *** uWSGI is running in multiple interpreter mode *** Sep 19 16:04:41.659423 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: spawned uWSGI master process (pid: 105623) Sep 19 16:04:41.659537 dv-cinder-858370-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 19 16:04:41.662797 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: spawned uWSGI worker 1 (pid: 105624, cores: 1) Sep 19 16:04:41.662797 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: spawned uWSGI worker 2 (pid: 105625, cores: 1) Sep 19 16:04:41.662797 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 16:04:42.338865 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 19 16:04:42.396938 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 19 16:04:43.441111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:04:43.441111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: from cryptography.utils import int_from_bytes Sep 19 16:04:43.441724 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:04:43.441724 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: from cryptography.utils import int_from_bytes Sep 19 16:04:43.515690 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:04:43.515690 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: from cryptography.utils import int_from_bytes Sep 19 16:04:43.516337 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:04:43.516337 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: from cryptography.utils import int_from_bytes Sep 19 16:04:45.936778 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 16:04:45.937347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 16:04:45.937602 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 16:04:45.937903 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 16:04:45.938153 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 16:04:45.938557 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.938879 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.939197 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.939403 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 16:04:45.939777 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.939868 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 16:04:45.939953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.940124 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.940250 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 16:04:45.940436 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.940529 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 16:04:45.940750 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.940834 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 16:04:45.941074 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941161 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941330 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941454 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941640 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941774 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.941960 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-858370-2-iscsi {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942091 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942275 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942437 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942603 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942749 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.942910 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943072 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943252 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943410 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943553 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943736 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943870 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.943995 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.944170 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.944307 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.944628 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-858370-2-iscsi {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.944722 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.944946 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.945895 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946045 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946166 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946282 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946427 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946531 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946607 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946734 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946907 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.946989 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947061 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947137 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947220 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947220 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947441 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947557 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947647 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947844 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.947967 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948154 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948258 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-858370-2-iscsi {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948473 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948580 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948787 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.948948 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949084 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949228 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949395 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949550 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949697 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.949843 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950005 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950133 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950318 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950731 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-858370-2-iscsi {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.950869 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951048 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951141 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951350 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951455 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951649 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951769 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.951978 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952071 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952277 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952364 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952579 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952685 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952883 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.952977 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953188 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953273 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953524 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953632 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953799 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.953885 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954100 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954196 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954474 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954675 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954787 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.954992 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955080 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955308 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955395 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955628 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955717 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955876 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.955962 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956121 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956262 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956437 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956557 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956753 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.956851 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957057 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957204 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957359 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957531 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957693 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.957850 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.958004 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.958169 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.958310 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.958445 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.958637 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.959242 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.959466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.959645 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.959766 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.959965 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960087 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960271 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960399 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960647 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960734 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.960988 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961089 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961309 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961394 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961652 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961744 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.57 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.961937 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.57 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.962023 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.962239 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.962516 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.962608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.962902 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963035 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963204 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963479 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963576 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963808 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.963896 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964113 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964199 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964413 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964498 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964714 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.964800 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965028 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965146 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.57 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965353 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965450 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.57 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965652 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965752 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.965897 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966022 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966193 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966340 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966513 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966674 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.966817 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967007 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967105 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967312 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967399 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967623 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967736 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.967925 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968058 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968295 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968396 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968548 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968635 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968849 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.968935 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969149 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969238 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969403 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969499 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969703 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.969805 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970008 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970311 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970420 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970637 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970723 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.970936 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971084 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971239 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971337 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971536 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971632 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971839 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.971937 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972136 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972221 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972435 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972535 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972735 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.972830 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973035 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973119 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973335 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973419 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973633 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973736 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.973935 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.974028 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.974236 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.974327 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975099 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975211 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975418 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975504 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975719 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.975804 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976020 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976116 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976327 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976459 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976618 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976721 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.976915 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977011 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977227 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977323 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977528 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977619 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977826 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.977919 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978122 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978216 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978445 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978579 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978747 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.978857 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979045 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979143 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979437 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979655 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979767 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.979928 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980025 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980229 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980325 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980532 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980627 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980841 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.980936 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981140 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981244 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981437 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981531 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981734 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.981827 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.982041 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.982175 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.982341 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.982472 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.982670 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.982757 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.982969 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.983065 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.983271 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.983358 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.983566 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.983651 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.983880 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.983980 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.984196 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.984283 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.984502 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.984596 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.984805 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.984891 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.985107 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.985204 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.985421 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:04:45.985514 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.985743 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.985830 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986059 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986145 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986396 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986523 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986722 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.986839 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987146 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987382 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987478 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987699 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.987793 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988008 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988175 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988339 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988521 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988650 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988829 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.988927 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989150 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989259 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989463 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989585 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989803 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.989891 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990156 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990246 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990491 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990581 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990811 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.990911 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991121 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991274 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991422 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991510 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991719 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.991821 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992024 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992111 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992350 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992445 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992659 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992748 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.992964 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993059 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993273 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993360 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993584 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993679 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.993900 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994073 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994210 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994297 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994540 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994645 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994849 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.994954 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995156 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995242 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995459 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995543 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995793 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.995889 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996105 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996202 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996418 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996504 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996730 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.996862 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997033 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997154 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997436 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997654 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997747 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.997965 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998087 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998266 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998352 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998605 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998882 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.998978 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999184 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999272 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999491 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999587 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999794 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:45.999890 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000198 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000513 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000728 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.000885 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001037 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001136 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001345 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001458 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001653 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001743 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.001957 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002257 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002343 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002581 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002700 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002890 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.002993 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003229 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003315 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003538 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003628 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003862 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.003967 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004172 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004269 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004473 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004570 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004795 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.004895 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005106 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005224 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005409 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005496 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005746 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.005837 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006056 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006155 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006392 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006483 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006704 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.006806 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007017 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007145 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007343 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007446 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007647 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007739 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.007951 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008284 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008401 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008599 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008795 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.008954 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009106 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009247 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009401 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009539 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009685 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009824 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.009959 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010096 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010246 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010424 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010562 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010706 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.010856 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011015 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011189 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011344 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011516 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011664 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011811 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.011988 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012138 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012287 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012435 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012583 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012760 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.012924 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013086 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013236 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013384 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013537 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013679 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013839 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.013989 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014131 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014280 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014475 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014649 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014812 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.014962 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015261 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015408 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015582 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015745 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.015895 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016194 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016505 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016669 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.016857 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017008 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017155 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017316 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017479 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017656 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017806 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.017969 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.018119 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.018281 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.018959 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019119 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019268 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019419 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019579 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019740 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.019905 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020203 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020351 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020502 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020665 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020816 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.020959 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.021112 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.021760 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.021978 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022148 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022308 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022481 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022631 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022782 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.022932 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023086 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023247 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023398 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023547 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023694 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.023842 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024105 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024266 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024415 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024566 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024732 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.024898 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025059 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025217 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025365 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025514 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025658 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025820 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.025969 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026120 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026267 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026442 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026592 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026760 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.026908 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027056 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027206 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027354 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027504 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027675 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027823 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.027972 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.028191 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.028351 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.028531 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.028700 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.028848 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029219 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029368 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029533 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029683 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029828 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.029990 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030139 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030288 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030480 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030628 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030775 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.030934 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031151 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031303 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031463 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031614 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031759 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.031928 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032132 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032282 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032446 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032593 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032737 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.032737 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033006 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033229 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033395 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033541 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033691 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033837 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.033986 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034154 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034302 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034487 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034637 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034810 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.034958 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035120 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035270 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035419 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035567 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035716 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.035864 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036027 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036176 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036321 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036482 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036632 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036796 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.036936 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.037744 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.037877 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038022 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038171 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038331 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038504 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038657 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038824 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.038985 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039143 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039288 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039437 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039586 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039735 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.039883 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040204 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040352 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040499 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040649 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040808 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.040970 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041119 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041259 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041419 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041568 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041727 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.041901 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.042059 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.042267 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.042565 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.042724 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.042887 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043050 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043200 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043409 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043561 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043720 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.043870 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.044277 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.044420 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.044579 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.044738 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.044895 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045064 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045214 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045372 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045520 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045663 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045809 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.045970 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.046125 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.046285 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.046541 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.046708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.046708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047015 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047229 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047380 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047529 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047677 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047833 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.047996 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048145 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048289 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048463 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048625 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048778 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.048928 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049088 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049235 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049445 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049606 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049769 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.049916 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050076 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050238 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050410 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050579 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050579 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.050866 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051013 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051160 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051389 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051543 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051692 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.051955 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.052106 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.052255 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.052527 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.052691 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.052860 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053010 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053172 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053320 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053542 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053542 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053832 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.053981 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054143 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054290 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054480 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054630 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054790 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.054959 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055125 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055276 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055425 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055585 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.055751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.056414 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.056579 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.056739 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.056904 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057053 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057198 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057344 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057492 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057651 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057800 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.057948 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058107 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058107 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058491 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058670 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058819 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.058993 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059160 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059308 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059527 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059681 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059831 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.059981 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060138 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060288 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060448 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060597 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060758 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.060758 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061047 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061215 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061376 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061510 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061655 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061801 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.061949 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062278 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062451 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062584 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062731 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.062881 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063832 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.063982 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064140 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064298 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064449 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064609 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064772 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.064931 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065079 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065226 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065374 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065518 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065683 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065833 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.065991 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066141 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066299 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066507 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066659 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066807 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.066954 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067101 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067260 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067402 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067551 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067708 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.067855 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068001 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068169 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068333 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068481 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068640 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068787 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.068935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069084 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069245 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069390 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069535 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069667 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069797 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.069941 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070092 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070241 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070413 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070559 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070708 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.070875 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071024 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071174 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071327 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071473 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071604 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071767 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.071928 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072085 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072232 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072232 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072500 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072733 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.072891 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073053 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073205 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073352 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073517 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073665 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073791 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.073939 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074081 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074215 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074363 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074581 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074704 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074839 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.074975 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075108 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075270 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075419 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075576 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075719 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.075866 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076011 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076170 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076315 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076461 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076607 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076751 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.076910 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.077069 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.077214 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.077214 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.077605 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.077754 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078017 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078166 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078313 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078481 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078643 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.078790 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079019 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079172 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079172 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079449 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.079895 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080038 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080196 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080753 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080901 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.080901 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081170 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081328 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081328 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081621 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081754 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.081893 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082034 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082180 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082354 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082603 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082748 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.082876 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.083038 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.083038 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.083348 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.083495 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.083650 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.084325 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.084467 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.084608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.084753 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.084893 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085028 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085151 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085281 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085423 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085548 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085684 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085684 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.085927 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086049 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086185 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086308 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086468 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086594 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086717 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086841 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.086975 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087099 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087230 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087354 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087477 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087598 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087733 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087869 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.087992 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088123 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088262 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088409 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088565 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088713 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.088860 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089008 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089153 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089310 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089441 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089567 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089706 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089839 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089968 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.089968 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090141 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090207 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090287 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090364 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090494 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090594 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090675 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090759 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.090940 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.091250 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.091347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.091554 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.091840 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.091927 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.092171 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.092415 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.092515 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.092769 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.093009 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.093101 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.093370 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.093569 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.093666 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.094098 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.094184 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.094436 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.094708 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.094795 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095041 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095299 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095386 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095651 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095867 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.095967 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.096273 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.096466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.096560 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.096980 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.097253 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.097349 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.097593 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.097828 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.097914 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.098197 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.098422 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.098531 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.098827 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.099154 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.099238 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.100083 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.100313 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.100415 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.100681 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.100895 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.101003 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.101311 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.101500 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.101632 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.102047 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.102132 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.102360 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.102690 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.102815 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.103105 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.103242 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.103417 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.103729 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.103814 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.104052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.104375 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.104471 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.104698 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.104970 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.105067 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.105304 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.105539 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.105636 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.105923 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.106057 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.106245 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.106585 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.106674 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.106901 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.107202 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.107289 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.107523 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.107794 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.107895 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.108145 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.108395 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.108481 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.108867 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.108961 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.109190 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.109500 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.109593 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.109824 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.110146 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.110275 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.110520 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.110773 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.110860 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111083 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111341 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111428 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111679 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111849 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.111998 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.112299 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.112385 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.112611 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.112834 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.112920 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.113212 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.113297 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.113517 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.113786 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.113875 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.114127 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.114265 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.114460 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.114775 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.114874 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115115 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115215 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115574 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115759 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.115958 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116048 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116316 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116402 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116667 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116841 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.116972 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.117274 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.117359 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.117576 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.117816 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.117911 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118159 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118246 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118501 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118594 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118858 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.118955 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119183 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119269 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119489 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119582 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119819 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.119921 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120133 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120219 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120446 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120532 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120756 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.120843 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121058 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121144 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121379 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121472 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121681 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.121969 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122056 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122277 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122389 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122600 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122745 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.122901 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123047 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123201 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123390 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123507 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123728 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.123818 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124045 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124130 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124382 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124484 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124696 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124833 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.124996 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125097 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125314 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125410 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125621 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125708 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.125923 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126052 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126223 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126389 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126545 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126757 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.126883 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127072 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127168 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127381 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127495 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127687 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127805 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.127993 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128130 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128304 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128446 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128610 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128767 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.128923 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129079 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129226 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129577 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129688 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129773 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.129853 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.130018 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.130151 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.130310 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.130485 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131156 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131309 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131473 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131614 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131787 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.131918 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.132104 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.132262 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.132564 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.132724 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.132877 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133020 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133185 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133335 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133509 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133609 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133825 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.133914 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134138 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134240 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134474 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134566 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134788 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.134892 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135105 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135201 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135418 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135506 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135730 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.135816 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136036 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136123 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136351 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136438 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136671 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136758 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.136987 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.137076 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.137309 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.137547 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.137634 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.137881 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138141 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138232 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138319 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138528 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138616 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138840 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.138927 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139152 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139247 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139551 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139777 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.139862 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140083 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140169 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140391 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140490 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140696 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.140789 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141007 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141100 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141325 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141463 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141632 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141736 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.141939 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142026 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142249 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142346 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142586 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142700 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142896 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.142987 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143208 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143299 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143533 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143633 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143846 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.143950 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.144155 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.144244 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.144466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.144554 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.144780 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145003 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145091 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145317 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145405 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145653 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145757 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.145973 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.146127 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.146329 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.146918 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147134 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147234 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147443 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147530 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147754 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.147841 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148062 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148148 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148388 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148477 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148715 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.148808 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149028 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149133 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149332 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149417 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149640 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149735 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.149950 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150045 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150256 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150511 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150603 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150830 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.150918 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.151166 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.151429 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.151617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.151777 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.151931 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152105 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152269 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152419 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152770 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.152919 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153078 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153225 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153368 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153512 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153684 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153814 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.153975 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154128 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154273 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154452 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154598 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154738 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.154888 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155031 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155200 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155492 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155650 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155807 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.155950 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156104 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156248 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156397 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156563 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156709 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156851 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.156986 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157135 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157274 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157434 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157593 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157741 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158035 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158514 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158616 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158697 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158812 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.158941 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159068 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159207 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159336 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159464 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159601 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.159862 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160003 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160130 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160258 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160427 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160539 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.160807 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.161052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.161468 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.161792 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162530 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162625 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162714 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162801 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162895 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.162977 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163055 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163220 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163330 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163472 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163577 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163813 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.163977 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164133 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164247 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164444 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164533 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164756 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.164842 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165061 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165242 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165399 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165490 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165654 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165794 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.165958 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.166103 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.166241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.166576 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.166668 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.166911 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167024 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167229 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167316 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167531 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167628 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.167852 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168020 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168155 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168243 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168458 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168555 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168759 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.168862 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.169331 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.169431 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.169643 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.169730 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.169951 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170055 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170258 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170344 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170657 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170753 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.170966 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171062 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171277 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171524 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171612 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171859 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.171974 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172174 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172261 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172481 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172568 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172790 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.172889 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173098 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173185 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173408 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173508 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173706 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.173801 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174019 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174126 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174321 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174448 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174658 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174749 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.174967 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175055 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175275 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175379 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175580 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175660 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175889 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.175987 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176192 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176279 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176498 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176585 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176801 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.176887 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.177111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.177207 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.177417 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.177641 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.178294 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.178451 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.178625 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.178728 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.178937 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.179249 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.179336 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.179592 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.179679 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.179940 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180035 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180229 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180316 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180542 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180652 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180853 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.180941 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181158 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181244 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181464 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181564 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181769 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.181872 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182076 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182175 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182409 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182514 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182731 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.182819 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.183040 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.183128 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.183340 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.183442 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.183703 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184027 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184163 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184315 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184418 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184628 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184714 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.184948 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185045 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185265 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185362 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185571 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185663 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185875 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.185979 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186185 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186274 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186513 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186605 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186822 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.186918 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187430 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187538 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187608 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187674 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187739 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.187804 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188012 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188101 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188316 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188413 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188714 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.188919 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189016 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189221 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189308 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189526 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189631 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189834 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.189922 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190134 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190222 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190460 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190571 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.190858 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191063 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191149 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191362 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191466 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191661 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191748 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.191960 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192048 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192269 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192378 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192569 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192656 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192869 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.192973 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193209 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193303 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193512 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193608 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193813 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.193920 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194111 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194200 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194440 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194538 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194743 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.194839 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195045 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195135 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195347 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195441 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195647 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195742 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.195947 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196077 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196279 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196367 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196585 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196686 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196897 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.196985 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.197222 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.197316 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.197563 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.197750 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.197838 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198023 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198142 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198219 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198480 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198633 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198802 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.198890 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199115 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199209 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199424 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199513 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199752 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.199847 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200056 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200144 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200365 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200453 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200670 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200757 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.200979 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201287 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201376 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201597 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201693 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.201915 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202021 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202228 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202414 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202554 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202664 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202874 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.202984 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.203186 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.203275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.203504 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.203591 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.203831 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204016 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204109 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204175 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204434 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204531 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204744 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.204849 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205051 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205140 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205360 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205455 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205669 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205758 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.205978 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206074 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206294 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206447 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206619 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206801 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.206978 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 16:04:46.207130 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.207285 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.207535 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.207859 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.208189 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.208499 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.208856 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.209182 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.209529 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.209841 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.210183 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.210531 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.210842 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.211159 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.211461 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:04:46.211700 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 16:04:46.368739 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO dbcounter [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Registered counter for database nova_api Sep 19 16:04:46.376462 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO dbcounter [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Registered counter for database nova_api Sep 19 16:04:46.394031 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_db.sqlalchemy.engines [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.401431 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_db.sqlalchemy.engines [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.401758 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writer thread running {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.408281 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writer thread running {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.460702 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.context [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c2c7f29-247d-4f27-aaaf-5b73a621e9cb(cell1) {{(pid=105625) load_cells /opt/stack/nova/nova/context.py:464}} Sep 19 16:04:46.464682 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:04:46.465155 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:04:46.466640 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.context [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c2c7f29-247d-4f27-aaaf-5b73a621e9cb(cell1) {{(pid=105624) load_cells /opt/stack/nova/nova/context.py:464}} Sep 19 16:04:46.466759 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:04:46.467500 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:04:46.467945 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:04:46.469968 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:04:46.470558 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:04:46.471030 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:04:46.472048 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO dbcounter [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Registered counter for database nova_cell0 Sep 19 16:04:46.472634 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:04:46.473384 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:04:46.473825 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:04:46.475936 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_concurrency.lockutils [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105624) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:04:46.476739 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO dbcounter [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Registered counter for database nova_cell1 Sep 19 16:04:46.477990 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO dbcounter [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Registered counter for database nova_cell0 Sep 19 16:04:46.482535 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO dbcounter [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Registered counter for database nova_cell1 Sep 19 16:04:46.484628 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_db.sqlalchemy.engines [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.486027 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_db.sqlalchemy.engines [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.492159 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_db.sqlalchemy.engines [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.492944 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writer thread running {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.494828 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_db.sqlalchemy.engines [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.496509 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writer thread running {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.499377 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writer thread running {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.639038 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writer thread running {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.640963 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO dbcounter [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Registered counter for database nova_cell0 Sep 19 16:04:46.649968 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_db.sqlalchemy.engines [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.650248 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO dbcounter [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Registered counter for database nova_cell0 Sep 19 16:04:46.655435 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writer thread running {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.659717 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG oslo_db.sqlalchemy.engines [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:04:46.664342 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writer thread running {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:04:46.996216 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: WARNING keystonemiddleware.auth_token [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:04:46.996804 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: WARNING keystonemiddleware.auth_token [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:04:47.003830 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: WARNING keystonemiddleware.auth_token [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:04:47.004052 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: WARNING keystonemiddleware.auth_token [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:04:47.355383 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: WARNING keystonemiddleware.auth_token [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:04:47.359724 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: WARNING keystonemiddleware.auth_token [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:04:47.361375 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: WARNING keystonemiddleware.auth_token [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:04:47.366043 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: WARNING keystonemiddleware.auth_token [None req-62b4901c-d233-450a-acfd-1678e23b8cb2 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:04:47.366976 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55815a4819c0 pid: 105625 (default app) Sep 19 16:04:47.369234 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:04:47.370467 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] 10.222.0.57 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001880 Sep 19 16:04:47.370896 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 1/1] 10.222.0.57 () {50 vars in 788 bytes} [Mon Sep 19 16:04:47 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:04:47.371934 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55815a4819c0 pid: 105624 (default app) Sep 19 16:06:12.686251 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:12.687209 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001505 Sep 19 16:06:12.687864 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 2/2] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:12 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:12.705209 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_api:SELECT=3 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:12.707207 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_cell0:SELECT=3 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:12.708227 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_cell1:SELECT=3 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:12.709168 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:13.710071 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-c23a9e52-7eb5-4c22-a8ba-0280bf99e2b0 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:13.744616 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-c23a9e52-7eb5-4c22-a8ba-0280bf99e2b0 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.048920 Sep 19 16:06:13.748407 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 1/3] 10.222.0.57 () {60 vars in 1292 bytes} [Mon Sep 19 16:06:12 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1051 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:16.557244 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:16.558255 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-8a50d6c6-40ed-4763-9803-e639a8a9ac25 None None] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001435 Sep 19 16:06:16.558861 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 3/4] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:16 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:06:17.075928 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-a32b035c-39ff-4916-b388-3b41ca3bd512 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "name": "cirros256", "disk": 1, "id": "c1", "OS-FLV-EXT-DATA:ephemeral": 0, "ram": 256, "rxtx_factor": 1.0, "vcpus": 1, "swap": 0}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:17.133674 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-a32b035c-39ff-4916-b388-3b41ca3bd512 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.567871 Sep 19 16:06:17.134484 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 2/5] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:16 2022] POST /compute/v2.1/flavors => generated 407 bytes in 570 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:06:17.148267 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=3 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:17.149149 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_cell0:SELECT=3 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:17.151870 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_cell1:SELECT=3 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:17.152707 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:17.365380 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-8e6ef187-f93d-4776-930e-6e2376038309 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:17.434188 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-8e6ef187-f93d-4776-930e-6e2376038309 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.291199 Sep 19 16:06:17.444712 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 4/6] 10.222.0.57 () {64 vars in 1336 bytes} [Mon Sep 19 16:06:17 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:20.246988 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-a32b035c-39ff-4916-b388-3b41ca3bd512 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:20.248324 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-a32b035c-39ff-4916-b388-3b41ca3bd512 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002079 Sep 19 16:06:20.249084 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 3/7] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:20 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:21.040222 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-bb52dc15-b742-4824-a52b-8cc26a70df62 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "d1", "disk": 5, "name": "ds512M", "vcpus": 1, "ram": 512, "os-flavor-access:is_public": true, "swap": 0, "rxtx_factor": 1.0}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:21.084798 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-bb52dc15-b742-4824-a52b-8cc26a70df62 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.826906 Sep 19 16:06:21.085606 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 5/8] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:20 2022] POST /compute/v2.1/flavors => generated 404 bytes in 829 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:06:21.098162 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-11214b2f-61e0-4e67-a16b-f6ab80e82698 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:21.142808 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-11214b2f-61e0-4e67-a16b-f6ab80e82698 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.048599 Sep 19 16:06:21.146791 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 4/9] 10.222.0.57 () {64 vars in 1336 bytes} [Mon Sep 19 16:06:21 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:06:23.963546 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-bb52dc15-b742-4824-a52b-8cc26a70df62 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:23.965881 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-bb52dc15-b742-4824-a52b-8cc26a70df62 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002853 Sep 19 16:06:23.966538 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 6/10] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:23 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:24.513288 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-d418fbf5-ae33-4115-9413-350068be9523 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "swap": 0, "vcpus": 1, "ram": 1024, "id": "d2", "disk": 10, "name": "ds1G", "rxtx_factor": 1.0}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:24.538239 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-d418fbf5-ae33-4115-9413-350068be9523 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.561808 Sep 19 16:06:24.538869 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 5/11] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:23 2022] POST /compute/v2.1/flavors => generated 404 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:06:24.551970 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-42972816-418e-420f-8678-916498b46c7c admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:24.586004 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-42972816-418e-420f-8678-916498b46c7c admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037745 Sep 19 16:06:24.587741 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 7/12] 10.222.0.57 () {64 vars in 1336 bytes} [Mon Sep 19 16:06:24 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:27.324799 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-d418fbf5-ae33-4115-9413-350068be9523 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:27.326643 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-d418fbf5-ae33-4115-9413-350068be9523 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002365 Sep 19 16:06:27.327665 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 6/13] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:27 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:27.767870 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-fce44424-807c-490b-b9fa-18ff0a3a0b15 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "vcpus": 2, "id": "d3", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "name": "ds2G", "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:27.798476 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-fce44424-807c-490b-b9fa-18ff0a3a0b15 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.464286 Sep 19 16:06:27.807019 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 8/14] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:27 2022] POST /compute/v2.1/flavors => generated 404 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:06:27.812690 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-35b9ec34-fc8d-493f-b7ce-03bce7602ccb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:27.843675 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-35b9ec34-fc8d-493f-b7ce-03bce7602ccb admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.035194 Sep 19 16:06:27.858439 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 7/15] 10.222.0.57 () {64 vars in 1336 bytes} [Mon Sep 19 16:06:27 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:30.589315 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-fce44424-807c-490b-b9fa-18ff0a3a0b15 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:30.590344 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-fce44424-807c-490b-b9fa-18ff0a3a0b15 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001482 Sep 19 16:06:30.591046 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 9/16] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:31.109734 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "vcpus": 4, "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "d4", "name": "ds4G", "disk": 20, "ram": 4096, "swap": 0}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:31.138707 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.538883 Sep 19 16:06:31.139997 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 8/17] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:30 2022] POST /compute/v2.1/flavors => generated 404 bytes in 541 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:06:31.156153 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-92c99cd7-0240-4b7a-9443-ee2245172b2b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:31.195680 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-92c99cd7-0240-4b7a-9443-ee2245172b2b admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.046185 Sep 19 16:06:31.196259 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 10/18] 10.222.0.57 () {64 vars in 1336 bytes} [Mon Sep 19 16:06:31 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:33.882071 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:33.883759 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002145 Sep 19 16:06:33.884306 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 9/19] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:33 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:34.486179 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-2a044f61-7640-4032-b3bc-e63153e4b2e8 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:34.512879 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-2a044f61-7640-4032-b3bc-e63153e4b2e8 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.620601 Sep 19 16:06:34.513428 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 11/20] 10.222.0.57 () {60 vars in 1292 bytes} [Mon Sep 19 16:06:33 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 622 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:06:37.473130 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:37.474680 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-39cd4ec0-aecc-4fcb-b6fb-f4cdf390abc2 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002914 Sep 19 16:06:37.475176 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 10/21] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:37 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:38.060641 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-3a866b2c-e64a-45d1-a34f-49b7e18d0238 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 512, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "id": "1", "disk": 1, "rxtx_factor": 1.0, "name": "m1.tiny", "swap": 0, "os-flavor-access:is_public": true}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:38.083930 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-3a866b2c-e64a-45d1-a34f-49b7e18d0238 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.602165 Sep 19 16:06:38.084662 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 12/22] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:37 2022] POST /compute/v2.1/flavors => generated 402 bytes in 604 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:06:38.098016 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-8b7d9e32-3233-4509-a946-50a12325bfba admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:38.127357 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-8b7d9e32-3233-4509-a946-50a12325bfba admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033630 Sep 19 16:06:38.127986 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 11/23] 10.222.0.57 () {64 vars in 1333 bytes} [Mon Sep 19 16:06:38 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:40.832922 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-3a866b2c-e64a-45d1-a34f-49b7e18d0238 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:40.833952 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-3a866b2c-e64a-45d1-a34f-49b7e18d0238 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001489 Sep 19 16:06:40.834361 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 13/24] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:41.255856 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-ac0647ba-b021-4986-b8ad-fb5f934af47a admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "disk": 20, "os-flavor-access:is_public": true, "name": "m1.small", "ram": 2048, "id": "2", "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "swap": 0}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:41.275710 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-ac0647ba-b021-4986-b8ad-fb5f934af47a admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.434923 Sep 19 16:06:41.276275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 12/25] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:40 2022] POST /compute/v2.1/flavors => generated 405 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:06:41.289711 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-c7fd3aa5-71e2-433c-9137-d236cb5e1b6a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:41.314999 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-c7fd3aa5-71e2-433c-9137-d236cb5e1b6a admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.028965 Sep 19 16:06:41.315613 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 14/26] 10.222.0.57 () {64 vars in 1333 bytes} [Mon Sep 19 16:06:41 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 30 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:43.986279 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-ac0647ba-b021-4986-b8ad-fb5f934af47a admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:43.988002 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-ac0647ba-b021-4986-b8ad-fb5f934af47a admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001733 Sep 19 16:06:43.988485 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 13/27] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:44.429129 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-990afa5d-4e19-4412-91a2-ba4a3de36a96 admin admin] Action: 'create', calling method: >, body: {"flavor": {"rxtx_factor": 1.0, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0, "id": "3", "disk": 40, "swap": 0, "os-flavor-access:is_public": true, "ram": 4096, "name": "m1.medium"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:44.450876 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-990afa5d-4e19-4412-91a2-ba4a3de36a96 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.455619 Sep 19 16:06:44.451449 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 15/28] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:43 2022] POST /compute/v2.1/flavors => generated 406 bytes in 457 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:06:44.463485 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-da42e32a-23d3-499b-90b4-23b48f65aa9b admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:44.468467 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=7,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:44.492310 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-da42e32a-23d3-499b-90b4-23b48f65aa9b admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032439 Sep 19 16:06:44.492894 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 14/29] 10.222.0.57 () {64 vars in 1333 bytes} [Mon Sep 19 16:06:44 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:47.342732 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-990afa5d-4e19-4412-91a2-ba4a3de36a96 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:47.343748 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-990afa5d-4e19-4412-91a2-ba4a3de36a96 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001502 Sep 19 16:06:47.344197 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 16/30] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:47 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:47.751177 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-12456f85-d4a5-4d19-88e9-1cb06bfa0eaf admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.large", "disk": 80, "vcpus": 4, "os-flavor-access:is_public": true, "ram": 8192, "rxtx_factor": 1.0, "id": "4"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:47.771036 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-12456f85-d4a5-4d19-88e9-1cb06bfa0eaf admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.420710 Sep 19 16:06:47.771600 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 15/31] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:47 2022] POST /compute/v2.1/flavors => generated 405 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:06:47.785977 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-1c3ae29b-ace6-416f-8375-7c7e0aadc7bb admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:47.792159 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:06:47.818862 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-1c3ae29b-ace6-416f-8375-7c7e0aadc7bb admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.037547 Sep 19 16:06:47.824192 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 17/32] 10.222.0.57 () {64 vars in 1333 bytes} [Mon Sep 19 16:06:47 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:06:50.632780 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-12456f85-d4a5-4d19-88e9-1cb06bfa0eaf admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:06:50.634159 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-12456f85-d4a5-4d19-88e9-1cb06bfa0eaf admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001832 Sep 19 16:06:50.634598 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 16/33] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:06:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:06:51.122704 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] Action: 'create', calling method: >, body: {"flavor": {"OS-FLV-EXT-DATA:ephemeral": 0, "id": "5", "ram": 16384, "swap": 0, "name": "m1.xlarge", "vcpus": 8, "disk": 160, "os-flavor-access:is_public": true, "rxtx_factor": 1.0}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:51.147017 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.505970 Sep 19 16:06:51.147755 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 18/34] 10.222.0.57 () {64 vars in 1283 bytes} [Mon Sep 19 16:06:50 2022] POST /compute/v2.1/flavors => generated 408 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:06:51.160973 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-c0c3a68f-a816-4a65-84ba-9b3c6a43d970 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:06:51.197816 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-c0c3a68f-a816-4a65-84ba-9b3c6a43d970 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.040480 Sep 19 16:06:51.206452 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 17/35] 10.222.0.57 () {64 vars in 1333 bytes} [Mon Sep 19 16:06:51 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:07:15.739795 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:15.740819 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 19 16:07:15.741225 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 19/36] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:07:15 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:15.752514 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_api:SELECT=14,nova_api:SAVEPOINT=2,nova_api:INSERT=3,nova_api:RELEASE=2 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:16.254445 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-3d913dd0-1f07-441a-b16b-2672c594c6b7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:16.273443 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-3d913dd0-1f07-441a-b16b-2672c594c6b7 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.525861 Sep 19 16:07:16.273921 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 18/37] 10.222.0.57 () {60 vars in 1292 bytes} [Mon Sep 19 16:07:15 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:17.590536 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:17.591544 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-e4aba9d6-2cde-4d35-a843-442d372526f2 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001473 Sep 19 16:07:17.591941 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 20/38] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:17 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:18.121699 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-49c55a5a-0420-45a7-9029-61a70b601b8c admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:18.127473 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.wsgi [None req-49c55a5a-0420-45a7-9029-61a70b601b8c admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-49c55a5a-0420-45a7-9029-61a70b601b8c admin admin] Request method failure captured: Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept: application/json Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept-Encoding: gzip, deflate Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Connection: keep-alive Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Content-Length: 0 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Host: 10.222.0.57 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-System-Scope: None Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: User-Agent: python-novaclient Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Auth-Token: gAAAAABjKJO1UlBu-VWwvYJjbNJrjyAsdqg1rpy1IpH54WA8l9-t7sL88XEJbz1ZnecvZpOx00L9fyoXOTSNoQZ_OzvTMj14_MaYkll2qbA2RxVT4MDUw97bMk4dNnlKB-UPIuqh0_TZ2rILmEFAVqBz0DrsPV3YQydxJ3Y7obd7NVhwpCPICkE Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Id: None Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Name: None Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Identity-Status: Confirmed Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Is-Admin-Project: True Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Id: default Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Name: Default Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Name: admin Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Role: member,reader,admin Sep 19 16:07:18.130708 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Roles: member,reader,admin Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant: admin Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Name: admin Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User: admin Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Id: default Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Name: Default Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Name: admin Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: method: > Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: exception: Flavor 42 could not be found. Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: action_args: {'id': '42'} Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:18.132225 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:18.133132 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.134563 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-49c55a5a-0420-45a7-9029-61a70b601b8c admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105624) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:18.135879 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-49c55a5a-0420-45a7-9029-61a70b601b8c admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.402006 Sep 19 16:07:18.135879 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 19/39] 10.222.0.57 () {60 vars in 1190 bytes} [Mon Sep 19 16:07:17 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 403 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:18.141982 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-4a04106a-08ec-4026-a6dd-159ac55e506a admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:18.146784 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:07:18.166267 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-4a04106a-08ec-4026-a6dd-159ac55e506a admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027793 Sep 19 16:07:18.166875 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 21/40] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:18 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 29 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:18.177992 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-1fae43d9-aa52-4bd7-bd0e-7537e8bfb445 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:18.200545 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-1fae43d9-aa52-4bd7-bd0e-7537e8bfb445 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.027119 Sep 19 16:07:18.201117 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 20/41] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:18 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 28 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:18.211438 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-aed71299-7f64-4e43-ab42-9a3fb11a42e6 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:18.217293 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.wsgi [None req-aed71299-7f64-4e43-ab42-9a3fb11a42e6 admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-aed71299-7f64-4e43-ab42-9a3fb11a42e6 admin admin] Request method failure captured: Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept: application/json Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept-Encoding: gzip, deflate Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Connection: keep-alive Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Content-Length: 0 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Host: 10.222.0.57 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-System-Scope: None Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: User-Agent: python-novaclient Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Auth-Token: gAAAAABjKJO1UlBu-VWwvYJjbNJrjyAsdqg1rpy1IpH54WA8l9-t7sL88XEJbz1ZnecvZpOx00L9fyoXOTSNoQZ_OzvTMj14_MaYkll2qbA2RxVT4MDUw97bMk4dNnlKB-UPIuqh0_TZ2rILmEFAVqBz0DrsPV3YQydxJ3Y7obd7NVhwpCPICkE Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Id: None Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Name: None Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Identity-Status: Confirmed Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Is-Admin-Project: True Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Id: default Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Name: Default Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Name: admin Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Role: member,reader,admin Sep 19 16:07:18.220617 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Roles: member,reader,admin Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant: admin Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Name: admin Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User: admin Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Id: default Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Name: Default Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Name: admin Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: method: > Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: exception: Flavor 42 could not be found. Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: action_args: {'id': '42'} Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:18.222194 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:18.223112 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:18.224622 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-aed71299-7f64-4e43-ab42-9a3fb11a42e6 admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105625) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:18.225960 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-aed71299-7f64-4e43-ab42-9a3fb11a42e6 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.014224 Sep 19 16:07:18.225960 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 22/42] 10.222.0.57 () {60 vars in 1190 bytes} [Mon Sep 19 16:07:18 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:19.455964 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-1fae43d9-aa52-4bd7-bd0e-7537e8bfb445 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:19.456981 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-1fae43d9-aa52-4bd7-bd0e-7537e8bfb445 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Sep 19 16:07:19.457379 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 21/43] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:19 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:19.991691 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-cd9da887-a595-45c4-a833-0344b32065f6 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:19.996749 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.wsgi [None req-cd9da887-a595-45c4-a833-0344b32065f6 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-cd9da887-a595-45c4-a833-0344b32065f6 admin admin] Request method failure captured: Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept: application/json Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept-Encoding: gzip, deflate Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Connection: keep-alive Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Content-Length: 0 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Host: 10.222.0.57 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-System-Scope: None Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: User-Agent: python-novaclient Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Auth-Token: gAAAAABjKJO3tLQ1ja7T-GGcGl8QH8kRQJKnqGj6g3rRh1-9HKtDPv8NXQBYMEl9Uizi727c1JyZb0ewhN6LWkSXbKBWYY2HzDikm-EJ3Cp2FktgHHC_dCOot6w9w9EUjKnhqsytCEEMLFSgFget78lKuOE_3uWb-iC8E8tMzMnnkx7PScmdvdU Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Id: None Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Name: None Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Identity-Status: Confirmed Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Is-Admin-Project: True Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Id: default Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Name: Default Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Name: admin Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Role: member,reader,admin Sep 19 16:07:19.998270 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Roles: member,reader,admin Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant: admin Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Name: admin Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User: admin Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Id: default Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Name: Default Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Name: admin Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: method: > Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: exception: Flavor 84 could not be found. Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: action_args: {'id': '84'} Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:19.999935 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:20.000953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.003241 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-cd9da887-a595-45c4-a833-0344b32065f6 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105625) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:20.005587 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-cd9da887-a595-45c4-a833-0344b32065f6 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.405628 Sep 19 16:07:20.005587 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 23/44] 10.222.0.57 () {60 vars in 1190 bytes} [Mon Sep 19 16:07:19 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 407 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:20.009771 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-7ca00d54-5f7c-4364-96a8-befddfdc85d2 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:20.026775 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-7ca00d54-5f7c-4364-96a8-befddfdc85d2 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020447 Sep 19 16:07:20.027369 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 22/45] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:20 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:20.037634 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-38536b4a-4141-4348-a723-d14c7c735aff admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:20.053678 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-38536b4a-4141-4348-a723-d14c7c735aff admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020143 Sep 19 16:07:20.054256 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 24/46] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:20 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:20.064128 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bd1a7e87-44e9-49af-b7d2-ed6a58200639 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:20.069228 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.wsgi [None req-bd1a7e87-44e9-49af-b7d2-ed6a58200639 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bd1a7e87-44e9-49af-b7d2-ed6a58200639 admin admin] Request method failure captured: Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept: application/json Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept-Encoding: gzip, deflate Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Connection: keep-alive Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Content-Length: 0 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Host: 10.222.0.57 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-System-Scope: None Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: User-Agent: python-novaclient Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Auth-Token: gAAAAABjKJO3tLQ1ja7T-GGcGl8QH8kRQJKnqGj6g3rRh1-9HKtDPv8NXQBYMEl9Uizi727c1JyZb0ewhN6LWkSXbKBWYY2HzDikm-EJ3Cp2FktgHHC_dCOot6w9w9EUjKnhqsytCEEMLFSgFget78lKuOE_3uWb-iC8E8tMzMnnkx7PScmdvdU Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Id: None Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Name: None Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Identity-Status: Confirmed Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Is-Admin-Project: True Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Id: default Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Name: Default Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Name: admin Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Role: member,reader,admin Sep 19 16:07:20.070764 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Roles: member,reader,admin Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant: admin Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Name: admin Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User: admin Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Id: default Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Name: Default Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Name: admin Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: method: > Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: exception: Flavor 84 could not be found. Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: action_args: {'id': '84'} Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:20.072275 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:20.073329 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:20.074809 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bd1a7e87-44e9-49af-b7d2-ed6a58200639 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105624) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:20.076053 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bd1a7e87-44e9-49af-b7d2-ed6a58200639 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011826 Sep 19 16:07:20.076053 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 23/47] 10.222.0.57 () {60 vars in 1190 bytes} [Mon Sep 19 16:07:20 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (2 switches on core 0) Sep 19 16:07:21.358998 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-38536b4a-4141-4348-a723-d14c7c735aff admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:21.360033 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-38536b4a-4141-4348-a723-d14c7c735aff admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001538 Sep 19 16:07:21.360447 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 25/48] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:21 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:21.890445 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-ec3e8ad3-07ca-4552-bcb7-f180886b4cbb admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:21.896893 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.wsgi [None req-ec3e8ad3-07ca-4552-bcb7-f180886b4cbb admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-ec3e8ad3-07ca-4552-bcb7-f180886b4cbb admin admin] Request method failure captured: Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept: application/json Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Accept-Encoding: gzip, deflate Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Connection: keep-alive Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Content-Length: 0 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Host: 10.222.0.57 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Openstack-System-Scope: None Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: User-Agent: python-novaclient Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Auth-Token: gAAAAABjKJO5-FK92dzQE8_wEBXfwVNjH9G_EPMNSzfumHjEa1SXID8iaYecHOR27-0C-maCfQO-KQVWWbXE5L-EY0Z0la9P-Y24k__AbLZuLuJ57VIaJXHGU3CQrZixX5mBmGlcclhgeegP7Wi3JWZhyaapJfT7XijV2uEhjqPMINDQj-lQi4g Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Id: None Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Domain-Name: None Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Identity-Status: Confirmed Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Is-Admin-Project: True Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Id: default Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Domain-Name: Default Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Project-Name: admin Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Role: member,reader,admin Sep 19 16:07:21.898422 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Roles: member,reader,admin Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant: admin Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-Tenant-Name: admin Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User: admin Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Id: default Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Domain-Name: Default Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: X-User-Name: admin Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: method: > Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: exception: Flavor 451 could not be found. Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: action_args: {'id': '451'} Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:21.899976 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:21.901065 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.902617 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-ec3e8ad3-07ca-4552-bcb7-f180886b4cbb admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105624) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:21.903862 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-ec3e8ad3-07ca-4552-bcb7-f180886b4cbb admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.396249 Sep 19 16:07:21.903862 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 24/49] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:07:21 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 397 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:21.909669 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-495732ed-d086-4d16-b3bc-f7d11ebc771d admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:21.925349 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-495732ed-d086-4d16-b3bc-f7d11ebc771d admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019310 Sep 19 16:07:21.925903 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 26/50] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 20 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:21.935882 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:21.951740 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.019352 Sep 19 16:07:21.952311 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 25/51] 10.222.0.57 () {60 vars in 1210 bytes} [Mon Sep 19 16:07:21 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 21 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:07:21.962670 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-dd895733-335b-4183-a102-437e41ec0ede admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:21.967806 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.wsgi [None req-dd895733-335b-4183-a102-437e41ec0ede admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-dd895733-335b-4183-a102-437e41ec0ede admin admin] Request method failure captured: Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept: application/json Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Accept-Encoding: gzip, deflate Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Connection: keep-alive Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Content-Length: 0 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Host: 10.222.0.57 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-Api-Version: compute 2.92 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Openstack-System-Scope: None Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: User-Agent: python-novaclient Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Auth-Token: gAAAAABjKJO5-FK92dzQE8_wEBXfwVNjH9G_EPMNSzfumHjEa1SXID8iaYecHOR27-0C-maCfQO-KQVWWbXE5L-EY0Z0la9P-Y24k__AbLZuLuJ57VIaJXHGU3CQrZixX5mBmGlcclhgeegP7Wi3JWZhyaapJfT7XijV2uEhjqPMINDQj-lQi4g Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Id: None Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Domain-Name: None Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Identity-Status: Confirmed Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Is-Admin-Project: True Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Openstack-Nova-Api-Version: 2.92 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Id: default Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Domain-Name: Default Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Project-Name: admin Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Role: member,reader,admin Sep 19 16:07:21.969338 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Roles: member,reader,admin Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5a804037a8734a35ad3ef5ec7e34d4c6"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5a804037a8734a35ad3ef5ec7e34d4c6"}]}] Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant: admin Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Id: 5a804037a8734a35ad3ef5ec7e34d4c6 Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-Tenant-Name: admin Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User: admin Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Id: default Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Domain-Name: Default Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Id: 18d4b3afb42f46999e6cf7b500999690 Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: X-User-Name: admin Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: method: > Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: exception: Flavor 451 could not be found. Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: action_args: {'id': '451'} Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:07:21.970997 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:21.972044 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: ERROR nova.api.openstack.wsgi Sep 19 16:07:21.973608 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-dd895733-335b-4183-a102-437e41ec0ede admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105625) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:07:21.974899 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-dd895733-335b-4183-a102-437e41ec0ede admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.011681 Sep 19 16:07:21.974899 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 27/52] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:07:21 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 19 16:07:23.231947 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:23.232964 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001447 Sep 19 16:07:23.233361 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 26/53] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:23 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:23.795996 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-4acb135f-0001-4d0c-8194-83609e45cd5c admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:07:23.818911 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-4acb135f-0001-4d0c-8194-83609e45cd5c admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.447115 Sep 19 16:07:23.819475 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 28/54] 10.222.0.57 () {64 vars in 1235 bytes} [Mon Sep 19 16:07:23 2022] POST /compute/v2.1/flavors => generated 404 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:07:25.068360 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:25.069471 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001622 Sep 19 16:07:25.069872 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 27/55] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:25 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:25.612277 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-a05b81c4-ac6f-48f5-a517-fb8c21539734 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:07:25.631335 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-a05b81c4-ac6f-48f5-a517-fb8c21539734 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.413722 Sep 19 16:07:25.631886 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 29/56] 10.222.0.57 () {64 vars in 1235 bytes} [Mon Sep 19 16:07:25 2022] POST /compute/v2.1/flavors => generated 405 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:07:26.890806 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:26.891814 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001465 Sep 19 16:07:26.892216 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 28/57] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:27.462069 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-c4432cf5-9b80-4662-85c7-db1f9b7bb8e8 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:07:27.483200 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-c4432cf5-9b80-4662-85c7-db1f9b7bb8e8 admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.452908 Sep 19 16:07:27.483748 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 30/58] 10.222.0.57 () {64 vars in 1235 bytes} [Mon Sep 19 16:07:27 2022] POST /compute/v2.1/flavors => generated 407 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:07:28.747043 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:28.748099 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001532 Sep 19 16:07:28.748500 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 29/59] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:28 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:29.305453 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-114e0a16-c106-4073-a8ce-acd4794e8d9d admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:07:29.325179 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-114e0a16-c106-4073-a8ce-acd4794e8d9d admin admin] 10.222.0.57 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.435316 Sep 19 16:07:29.325729 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 31/60] 10.222.0.57 () {64 vars in 1235 bytes} [Mon Sep 19 16:07:28 2022] POST /compute/v2.1/flavors => generated 409 bytes in 437 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:07:30.575014 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:30.576137 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001504 Sep 19 16:07:30.576452 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 30/61] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:07:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:07:31.130489 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-aec9390b-8be1-4b78-b4df-a22c40aeccc6 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:07:31.151621 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-aec9390b-8be1-4b78-b4df-a22c40aeccc6 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.427162 Sep 19 16:07:31.152160 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 32/62] 10.222.0.57 () {60 vars in 1202 bytes} [Mon Sep 19 16:07:30 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:08:28.754551 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:28.755788 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-226e24c2-8b67-4149-b42d-d156eabc5bb7 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001627 Sep 19 16:08:28.756218 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 31/63] 10.222.0.57 () {54 vars in 931 bytes} [Mon Sep 19 16:08:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:08:28.767185 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:29.162123 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-889ff9fc-de41-4cfd-bd64-98db4e0846c6 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:29.182615 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-889ff9fc-de41-4cfd-bd64-98db4e0846c6 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.419979 Sep 19 16:08:29.183157 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 33/64] 10.222.0.57 () {60 vars in 1292 bytes} [Mon Sep 19 16:08:28 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:08:29.204428 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_api:SELECT=14 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:08:29.208190 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-829c3833-f84d-451c-a708-2f17e2b9940b admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:29.215843 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-829c3833-f84d-451c-a708-2f17e2b9940b admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.013354 Sep 19 16:08:29.216380 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 32/65] 10.222.0.57 () {60 vars in 1283 bytes} [Mon Sep 19 16:08:29 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:08:29.228689 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-7bd59d3e-26bd-451b-b92b-dae59bade4fb admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:29.236248 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-7bd59d3e-26bd-451b-b92b-dae59bade4fb admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011133 Sep 19 16:08:29.236798 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 34/66] 10.222.0.57 () {60 vars in 1286 bytes} [Mon Sep 19 16:08:29 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 12 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:08:29.248512 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:29.256502 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011557 Sep 19 16:08:29.257028 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 33/67] 10.222.0.57 () {60 vars in 1286 bytes} [Mon Sep 19 16:08:29 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:08:29.269443 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-98b0d0db-4bce-4ded-bf08-91c6d18b4735 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:08:29.276889 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-98b0d0db-4bce-4ded-bf08-91c6d18b4735 admin admin] 10.222.0.57 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011723 Sep 19 16:08:29.277432 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 35/68] 10.222.0.57 () {60 vars in 1283 bytes} [Mon Sep 19 16:08:29 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:11:26.331223 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:26.332235 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001480 Sep 19 16:11:26.332653 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 34/69] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:11:26 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:11:26.472336 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=6 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:26.640223 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:26.644927 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:26.645402 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:26.645825 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:26.646571 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:26.647017 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:26.647429 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:26.679839 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-4e8478ab-4389-406b-8080-8fb7612f9af5 admin admin] 10.222.0.57 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.208612 Sep 19 16:11:26.680391 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 36/70] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:11:26 2022] GET /compute/v2.1/os-services => generated 791 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:11:27.762927 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:27.763938 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001494 Sep 19 16:11:27.764504 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 35/71] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:11:27 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:11:27.958653 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:27.962039 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:27.963407 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:27.963898 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:27.964655 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:27.965221 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:27.965705 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:27.987020 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-452a5fce-fe8e-4bc0-99c6-15dfcbaf51e3 admin admin] 10.222.0.57 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.083578 Sep 19 16:11:27.987653 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 37/72] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:11:27 2022] GET /compute/v2.1/os-services => generated 791 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:11:39.504195 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:39.505203 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001419 Sep 19 16:11:39.505603 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 36/73] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:11:39 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:11:39.650506 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_cell0:SELECT=4 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:39.651234 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_cell1:SELECT=4 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:39.651953 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG dbcounter [-] [105625] Writing DB stats nova_api:SELECT=8 {{(pid=105625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:11:39.701597 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:39.705080 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:39.705555 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:39.706008 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:39.706748 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:39.707189 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:39.707601 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:39.732084 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-b568f53c-82d8-4a82-b14f-3da119d85814 admin admin] 10.222.0.57 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.086252 Sep 19 16:11:39.732570 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 38/74] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:11:39 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:11:40.824773 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG nova.api.openstack.wsgi [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] Calling method '>' {{(pid=105624) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:40.826339 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: INFO nova.api.openstack.requestlog [None req-bac81a57-f051-4b29-a420-52a953ecf086 admin admin] 10.222.0.57 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001945 Sep 19 16:11:40.826769 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: [pid: 105624|app: 0|req: 37/75] 10.222.0.57 () {56 vars in 1074 bytes} [Mon Sep 19 16:11:40 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:11:41.020778 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG nova.api.openstack.wsgi [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Calling method '>' {{(pid=105625) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:11:41.024233 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:41.024730 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:41.025155 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:41.025889 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:11:41.026333 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:11:41.026769 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: DEBUG oslo_concurrency.lockutils [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105625) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:11:41.048941 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: INFO nova.api.openstack.requestlog [None req-224d5f5c-161c-4e72-9ffa-dd65da9cfe89 admin admin] 10.222.0.57 "GET /compute/v2.1/os-services" status: 200 len: 1035 microversion: 2.92 time: 0.081506 Sep 19 16:11:41.049428 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: [pid: 105625|app: 0|req: 39/76] 10.222.0.57 () {60 vars in 1193 bytes} [Mon Sep 19 16:11:40 2022] GET /compute/v2.1/os-services => generated 1035 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:12:38.288439 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: Mon Sep 19 16:12:38 2022 - graceful shutdown triggered... Sep 19 16:12:38.288190 dv-cinder-858370-2-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 19 16:12:38.289413 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: Gracefully killing worker 1 (pid: 105624)... Sep 19 16:12:38.289791 dv-cinder-858370-2-iscsi devstack@n-api.service[105625]: Gracefully killing worker 2 (pid: 105625)... Sep 19 16:12:38.416337 dv-cinder-858370-2-iscsi devstack@n-api.service[105624]: DEBUG dbcounter [-] [105624] Writing DB stats nova_api:SELECT=4 {{(pid=105624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:12:39.288419 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: worker 1 buried after 1 seconds Sep 19 16:12:39.288658 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: worker 2 buried after 1 seconds Sep 19 16:12:39.288658 dv-cinder-858370-2-iscsi devstack@n-api.service[105623]: goodbye to uWSGI. Sep 19 16:12:39.290084 dv-cinder-858370-2-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 19 16:12:39.290702 dv-cinder-858370-2-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 19 16:12:39.293479 dv-cinder-858370-2-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 19 16:12:39.303793 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 19 16:12:39 2022] *** Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: nodename: dv-cinder-858370-2-iscsi Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: machine: x86_64 Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: clock source: unix Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: pcre jit disabled Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: detected number of CPU cores: 4 Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: current working directory: / Sep 19 16:12:39.305754 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: detected binary path: /usr/bin/uwsgi-core Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: your processes number limit is 47680 Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: your memory page size is 4096 bytes Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: detected max file descriptor number: 2048 Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: lock engine: pthread robust mutexes Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: thunder lock: enabled Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 19 16:12:39.306801 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 19 16:12:39.354030 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: Python main interpreter initialized at 0x5594d4b0c9c0 Sep 19 16:12:39.354030 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: python threads support enabled Sep 19 16:12:39.354030 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: your server socket listen backlog is limited to 100 connections Sep 19 16:12:39.354030 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: your mercy for graceful operations on workers is 90 seconds Sep 19 16:12:39.354571 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: mapped 403077 bytes (393 KB) for 2 cores Sep 19 16:12:39.354571 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: *** Operational MODE: preforking *** Sep 19 16:12:39.354783 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: *** uWSGI is running in multiple interpreter mode *** Sep 19 16:12:39.354783 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: spawned uWSGI master process (pid: 120646) Sep 19 16:12:39.354995 dv-cinder-858370-2-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 19 16:12:39.356119 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: spawned uWSGI worker 1 (pid: 120663, cores: 1) Sep 19 16:12:39.356119 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: spawned uWSGI worker 2 (pid: 120664, cores: 1) Sep 19 16:12:39.356119 dv-cinder-858370-2-iscsi devstack@n-api.service[120646]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 19 16:12:40.018230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 19 16:12:40.023984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 19 16:12:41.188460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:12:41.188460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: from cryptography.utils import int_from_bytes Sep 19 16:12:41.189074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:12:41.189074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: from cryptography.utils import int_from_bytes Sep 19 16:12:41.216186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:12:41.216186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: from cryptography.utils import int_from_bytes Sep 19 16:12:41.216730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 19 16:12:41.216730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: from cryptography.utils import int_from_bytes Sep 19 16:12:43.610430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 16:12:43.611018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 16:12:43.611255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 16:12:43.611611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 16:12:43.611864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 16:12:43.612260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.612588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.612911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.613229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.613566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.613880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.614204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.614564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.614901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.615164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.615480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.615985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-858370-2-iscsi {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.616315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.616628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.616941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.617280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.617576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.617905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.618212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.618762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.620801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.621100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.621429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.621750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.622048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.622492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-858370-2-iscsi {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.623000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.623490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.623806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.624463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.624758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.625050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.625375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.625712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.626020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.626353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.626708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.627023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.627351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.627639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.627950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.628265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.628587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.628892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.629203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.629453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.629760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.630083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.630429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.630760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.631062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.631269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.631578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.631896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.632204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.632510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.632813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.633120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.633422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.633725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.634026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.634328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.634689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.635005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.635329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.635636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.635955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.636415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.57 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.636666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.57 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.636971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.637318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.637641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.637951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.638270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.638622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.638932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.639241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.639545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.639869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.640208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.640514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.640767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.641069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.641374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.641694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.642000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.642304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.642661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.642964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.643269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.643578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.643878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.644181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.644484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.644790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.645093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.645396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.645702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.646002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.646331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.646721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.646981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.647299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.647587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.647893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.648215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.648519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.648821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.649147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.649454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.649758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.650074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.650403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.650724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.651046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.651346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.652178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.652517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.652829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.653128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.653431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.653733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.654033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.654337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.654664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.654967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.655311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.655639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.655964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.656275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.656580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.656884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.657204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.657544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.657939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.658290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.658661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.659002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.659334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.659652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.659961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.660276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.660604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.660913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.661222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.661531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.661846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.662166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.662526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.662845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.663160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.663478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.663788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.664103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.664432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.664756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.665073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.665415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.665766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.666084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.666449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.666781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.667093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.667999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.668326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.668676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.668987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.669311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.669619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.669930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.670254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.670859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.671168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.671480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.671788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.672097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.672439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.672754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.673077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.673393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.673699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.674015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.674326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.674676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.674985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.675294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.676491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.676491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.676491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.676491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.677142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.677142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.677434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.677966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.678142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.678287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.678640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.678939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.679246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.679605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.680053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.680223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.680511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.680781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.681081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.681408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.681716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.682018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.682329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.682665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.683094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.683658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.684177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.684582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 19 16:12:43.685281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.685454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 19 16:12:43.685610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.685762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 19 16:12:43.685915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.686060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 19 16:12:43.686213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.686366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 19 16:12:43.686553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.686730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.686886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.687040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.687274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.687420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.687791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.687955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.688123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.688284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.688441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.688969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.689133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.689280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.689443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.689598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.689752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.689898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.690041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.690450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.690636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.690817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.691079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.691079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.691257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-858370-2-iscsi {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.691349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.691581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.691748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.691882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.691971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.692064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.692203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.692348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.692479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.692616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.692754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.692844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.692936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.693126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.693394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.693656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.693830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.693920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.694004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.694103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.694196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.694280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.694402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.694550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.694720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.694880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.695029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.695197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.695332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.695531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.695631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.695846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.695947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.696241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.696331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-858370-2-iscsi {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.696552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.696656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.696869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.696962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.697177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.697275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.697501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.697602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.697809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.697901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.698128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.698266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.698455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.699335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.699431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.699533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.699636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.699734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.699947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.700054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.700282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.700432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.700586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.700680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.700895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.701000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.701207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.701302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.701515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.701608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.701821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.701928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.702128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.702223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.702459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.702567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.702771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.702871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.703025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.703126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.703337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.703429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.703667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.703762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.703986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.704082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.704292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.704392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.704554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.704656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.704807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.704910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.705134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.705250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.705459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.705562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.705773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.705874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.706085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.706186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.706449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.706599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.706732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.706846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.707041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.707146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.707347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.707439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.707652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.707743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.707958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.708060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.708260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.708358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.708566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.708677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.708878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.708980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.709262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.709380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.709501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.709617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.709802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.709897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.57 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.710112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.710204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.57 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.710908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.711021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.711126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.711215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.711360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.711460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.711546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.711644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.711731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.711870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.711960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.712054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.712249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.712359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.712569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.712713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.712872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.713014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.713177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.713392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.713485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.713578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.713792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.713951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.714250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.714983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.715115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.715281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.715394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.715514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.715628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.715771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.715894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.716055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.716189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.716305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.716478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.716600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.716798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.716925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.717109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.717236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.717420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.717540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.717726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.717852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.718034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.718153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.718344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.719940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.720062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.720182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.720301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.720424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.720554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.720678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.720798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.720916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.721029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.721153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.721153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.721375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.721504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.721627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.721747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.721868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.722016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.722175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.722316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.722477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.722611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.722810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.722966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.723113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.723281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.723435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.723435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.723879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.724046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.724205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.724309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.724405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.724499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.724618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.724773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.724886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.724994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.724994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.725176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.725330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.725433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.725519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.725603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.725686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.725815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.725907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.725989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.726061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.726160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.726252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.726344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.726539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.726668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.726851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.726986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.727152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.727271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.727449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.727546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.727776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.727894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.728052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.728222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.728358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.728461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.728668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.728778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.728959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.729062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.729263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.729355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.729682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.729820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.729904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.729991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.730209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.730298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 19 16:12:43.730539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.730653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.730865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.730958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.731888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.732892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.733841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.734815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.735984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.736923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.737894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.738877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.739987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.740188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.740311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.740505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.740742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.740914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.741855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.742896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.743905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.744935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.745965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.746858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.747869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.748883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.749027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.749166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.749302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.749468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.749564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.750962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.751991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.752988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.753204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.753284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.753662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.753971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.754289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.754572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.754778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.754930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.755945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.756991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.757847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.758040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.758190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.758344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.759955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.760879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.761579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.761678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.761871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.761963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.762964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.763145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.763345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.763455 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.763557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.764796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.765755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.766848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.767926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.768980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.769920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.770812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.771950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.772973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.776395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.776514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.776727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.776727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.777987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.778919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.779983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.780717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.781997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.782901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783708 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.783847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.784977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.785947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.786921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.787943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.788849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.789816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.790787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.791962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.792053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.792301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.792386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.792623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.792943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.793277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.793436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.793582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.793725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.793881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.794922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.795067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.795211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.795354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.795505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.795981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.796942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.797962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.798936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.799955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.800911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.801945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.802146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.802251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.802718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.802843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.802949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.803881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.804952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.805975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.806971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.807956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.808897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.809803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.810804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.811902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.812945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.813854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814489 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.814792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.815954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.816883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.817895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.818819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.819936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.820959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.821878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.822093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.822190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.822414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.822655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.822749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.823006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.823104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.823316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.823407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824171 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.824894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.825823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.826984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.827989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.828930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.829863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.830975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.831933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.832969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.833931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.834965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.835963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.836281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.836397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.836596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.836705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.836932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.837981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.838196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.838539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.838718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.838872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.839035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.840000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.840226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.840563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.840880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.840998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.841936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.842134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.842238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.842511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.842846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.843176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.843351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.843531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.843696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.843934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.844908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.845932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.846913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.847838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.848866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.849892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.850925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.851900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.852884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.853835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.854889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.855894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.856814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.857959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.858901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.859930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.860838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.861909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.862834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.863987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.864086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.864300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.864443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.864638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.864729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.865721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.865872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.865872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.866992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.867918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.868840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.869962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.870591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.870591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.870825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.870935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.871880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.872948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.873878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.874835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.875973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.876936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.877897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.878918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.879156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.879284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.879492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.879612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.879753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120664) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 16:12:43.879879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.880136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.880466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.880749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.881070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.881388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.881705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.882027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.882346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.882705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.883039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.883376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.883726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.884052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.884377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.884701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.885023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.885390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.885711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.886574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.886889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.887213 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.887536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.887853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.888341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.888471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.888814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.889107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.889798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.889798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.890054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.890364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.890736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.891046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.891367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.891689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.892010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.892563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.892882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.893225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.893564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.893898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.894222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.894572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.894942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.895291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.895636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.895966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.896330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.896682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.897041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.897397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.897710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.898037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.898354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.898949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.899277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.899604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.899938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.900367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.900591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.900935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.901222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.901537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.902453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.902748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.903068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.903381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.903692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.904010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.904325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.904634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.904948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.905257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.905589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.905904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.906215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.906551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.906870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.907182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.907495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.907806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.908117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.908437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.908974 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.909220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.909457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.909775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.910132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.910472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.910974 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.911112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.911587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.911727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.912501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.912501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.912817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.912980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.913687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.913687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.913989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.914745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.915097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.915429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.915774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.916115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.916431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.916745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.917058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.917420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.917741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.918061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.918418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.918717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.919109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.919361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.919659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.919978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.920290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.920850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.921210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.921476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.921792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.922106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.929445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.932040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.934148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.936028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.939625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.941570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.945226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.948000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.949567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO dbcounter [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Registered counter for database nova_api Sep 19 16:12:43.949717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.949717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.949717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.949717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.949717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.950530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.950530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.950986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.952667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.952667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.952667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.952667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.952667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.953498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.953498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.953826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.954615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.954615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.954945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.955183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.955512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.955869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.956210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.956538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.957015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.957186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 19 16:12:43.957407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120663) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 19 16:12:43.974504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_db.sqlalchemy.engines [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:43.981517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writer thread running {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.020688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO dbcounter [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Registered counter for database nova_api Sep 19 16:12:44.038648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.context [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c2c7f29-247d-4f27-aaaf-5b73a621e9cb(cell1) {{(pid=120664) load_cells /opt/stack/nova/nova/context.py:464}} Sep 19 16:12:44.041595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_db.sqlalchemy.engines [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.042782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:12:44.043195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:12:44.044922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:12:44.045545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:12:44.046237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:12:44.048289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:12:44.049695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writer thread running {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.050331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO dbcounter [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Registered counter for database nova_cell0 Sep 19 16:12:44.054534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO dbcounter [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Registered counter for database nova_cell1 Sep 19 16:12:44.063576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_db.sqlalchemy.engines [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.065507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_db.sqlalchemy.engines [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.071634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writer thread running {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.077663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writer thread running {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.110024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.context [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),0c2c7f29-247d-4f27-aaaf-5b73a621e9cb(cell1) {{(pid=120663) load_cells /opt/stack/nova/nova/context.py:464}} Sep 19 16:12:44.114100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:12:44.115172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:12:44.116171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:12:44.116891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:12:44.117336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:12:44.119371 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:12:44.121895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO dbcounter [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Registered counter for database nova_cell0 Sep 19 16:12:44.125986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO dbcounter [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Registered counter for database nova_cell1 Sep 19 16:12:44.133432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_db.sqlalchemy.engines [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.134922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_db.sqlalchemy.engines [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.143048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writer thread running {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.143391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writer thread running {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.223036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO dbcounter [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Registered counter for database nova_cell0 Sep 19 16:12:44.229735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_db.sqlalchemy.engines [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120664) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.234233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writer thread running {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.290642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO dbcounter [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Registered counter for database nova_cell0 Sep 19 16:12:44.299419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_db.sqlalchemy.engines [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120663) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 19 16:12:44.304030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writer thread running {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 19 16:12:44.561605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: WARNING keystonemiddleware.auth_token [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:12:44.568850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: WARNING keystonemiddleware.auth_token [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:12:44.627897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: WARNING keystonemiddleware.auth_token [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:12:44.635385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: WARNING keystonemiddleware.auth_token [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:12:44.929278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: WARNING keystonemiddleware.auth_token [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:12:44.937151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: WARNING keystonemiddleware.auth_token [None req-6eaabc95-8c54-4675-a7b3-3c669d4189f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:12:44.942701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5594d4b0c9c0 pid: 120664 (default app) Sep 19 16:12:44.983262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: WARNING keystonemiddleware.auth_token [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 19 16:12:44.989296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: WARNING keystonemiddleware.auth_token [None req-2809ddce-d2e1-42e9-930d-b0b0ea21d4bf None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 19 16:12:44.994712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5594d4b0c9c0 pid: 120663 (default app) Sep 19 16:13:37.114688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:37.120899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:37.125426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:37.126913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:39.596627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-883f5e30-7090-4672-8d76-b8c27c5f7d6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: .version_select at 0x7fadc085f4c0>, body: {"keypair": {"name": "tempest-keypair-2115728446", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBI6mlSItuBvkErnADT87vb/aQjTdrpCHXlacaSsY5Tq1NNdHZk/0Z4kj3AaQwsCKqin2vZwG4fCRxWIEfb+/TCyGU1Y7UZr/wvbl7THh9ZiVc+XeqxF2i1PITJOWjs+MeA=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:39.794168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-883f5e30-7090-4672-8d76-b8c27c5f7d6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:39.810291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-883f5e30-7090-4672-8d76-b8c27c5f7d6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:39.932496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-883f5e30-7090-4672-8d76-b8c27c5f7d6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 2.834347 Sep 19 16:13:39.933676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1/1] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:13:37 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 2837 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:13:43.274924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:43.276722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:43.278037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:43.290143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:44.044365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-266118646", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "networks": [{"uuid": "3d13a155-5005-43b7-ac5a-30d519aab3f9"}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:45.980281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3af13161-a248-4756-8d31-cafa7c60afad tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'create', calling method: .version_select at 0x7fadc085e430>, body: {"keypair": {"name": "tempest-keypair-132643020", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBP6cnI1nAZJ9dQEDvpJGb6IY5DTWD2nOVpX/+BCaV906bnmfEyDsCSOOKrKNWDGpVSItNe9v1XM8ZPZFWLW/SiS/rO81/k22di8sVQ4jAWdfNCNBmE3PlBJP720gcAtVaw=="}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:46.116813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-3af13161-a248-4756-8d31-cafa7c60afad tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'key_pairs'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:46.137325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-3af13161-a248-4756-8d31-cafa7c60afad tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for user 754c1b5cb47746cf93c52d6dc0b8fa01 and project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'key_pairs'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:46.250715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3af13161-a248-4756-8d31-cafa7c60afad tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 2.987467 Sep 19 16:13:46.253019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1/2] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:13:43 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 2993 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:13:47.005154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-128799450", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:47.046098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] validate_networks() for [('3d13a155-5005-43b7-ac5a-30d519aab3f9', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:47.521405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] validate_networks() for [('32b57894-920f-4402-a64b-7ff50db77230', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:49.838133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:13:50.010348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:13:50.011712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:13:50.013014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:13:50.014209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:13:50.015314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:13:50.076167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:50.078301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:50.081669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:50.085873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:50.088154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:50.092500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:50.166736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Getting quotas for project 34baf55057ee42ffbd099de2d65544a1. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:50.178048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Getting quotas for user 428a95af27d2468bb39391ca3eefea40 and project 34baf55057ee42ffbd099de2d65544a1. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:50.207203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:13:50.208764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:13:50.209685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:13:50.209972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:13:50.210760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:13:50.213107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:13:50.223605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:13:50.269414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:50.270626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:50.271094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:50.271902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:50.273048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:50.273878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:50.360137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:50.383120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for user 8cd9c727c0b444cb83497ea4f78e24d0 and project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:50.420577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:13:50.426255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:13:50.485867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cf24f1fa-b0e4-4c5f-b35b-776d99e7bbbf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 6.988805 Sep 19 16:13:50.491342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 2/3] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:13:43 2022] POST /compute/v2.1/servers => generated 380 bytes in 6996 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:13:50.633587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c6a2ebc9-04c7-4008-893e-7bd551bfc573 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.378272 Sep 19 16:13:50.635433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 2/4] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:13:46 2022] POST /compute/v2.1/servers => generated 383 bytes in 4381 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:13:50.671327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1263344516", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "145a1b1d-9f41-428a-98d1-15b34e55a4ad"}], "security_groups": [{"name": "tempest-securitygroup--833765472"}], "key_name": "tempest-keypair-2115728446", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:50.744569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a57e46b2-8834-4bae-967a-78eeaa47256b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: .version_select at 0x7fadc0679e50>, body: {"keypair": {"name": "tempest-keypair-1741274330", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBItISHU71TkNIDUrp+UDByUJEZKagmXh3bO5gMikQA8HVftq5zShBuMko5L5Vtr1UX/3QKCs/hu/cQoTCN1kHUciHaHQUlbX5xPpkC8W1+Ta5+wb6773xAxeN3PSHQ5OIA=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:50.842572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a57e46b2-8834-4bae-967a-78eeaa47256b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:50.855587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a57e46b2-8834-4bae-967a-78eeaa47256b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for user bd190d8e252342ca9232f7a0c966c715 and project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:50.949052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a57e46b2-8834-4bae-967a-78eeaa47256b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.452795 Sep 19 16:13:50.950733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 3/5] 10.222.0.57 () {62 vars in 1200 bytes} [Mon Sep 19 16:13:50 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 457 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 19 16:13:50.988825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-520e759c-bba4-46a6-b4cb-4eae15ed727e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:50.989700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-520e759c-bba4-46a6-b4cb-4eae15ed727e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:51.115699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] validate_networks() for [('145a1b1d-9f41-428a-98d1-15b34e55a4ad', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:51.750137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-520e759c-bba4-46a6-b4cb-4eae15ed727e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:51.783478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-520e759c-bba4-46a6-b4cb-4eae15ed727e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1214 microversion: 2.1 time: 0.831512 Sep 19 16:13:51.799914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 4/6] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:50 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1214 bytes in 848 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:51.804908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-abcf06e0-0ea5-45ec-a521-33e7c81182af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:51.805535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-abcf06e0-0ea5-45ec-a521-33e7c81182af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:52.106134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-abcf06e0-0ea5-45ec-a521-33e7c81182af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:52.115108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-abcf06e0-0ea5-45ec-a521-33e7c81182af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1221 microversion: 2.1 time: 0.313963 Sep 19 16:13:52.115714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 5/7] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:51 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1221 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:52.360073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-64690118", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:52.637229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] validate_networks() for [('32b57894-920f-4402-a64b-7ff50db77230', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:53.204733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:13:53.205554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:13:53.206488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:13:53.207424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:13:53.208138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:13:53.287481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:53.291147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:53.291735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:53.298470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:53.299108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:53.299571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:53.375551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:53.394690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:53.468774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:13:53.478128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:13:53.682512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-46a473a4-2d7b-49ea-a459-c4ec3ee91e80 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 3.045310 Sep 19 16:13:53.692254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 3/8] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:13:50 2022] POST /compute/v2.1/servers => generated 401 bytes in 3056 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:13:53.715001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:53.716188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:53.755349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:53.759300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:53.759979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:53.762293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:53.764229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:53.765290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:54.518228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:54.588495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8d86a4cd-de02-4269-9b06-51ef4a7b8d7e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1218 microversion: 2.1 time: 0.892523 Sep 19 16:13:54.609363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 4/9] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:53 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1218 bytes in 914 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:54.619168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fb3b6b67-5647-4640-9022-a74959993d19 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:54.620341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fb3b6b67-5647-4640-9022-a74959993d19 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:54.807735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fb3b6b67-5647-4640-9022-a74959993d19 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:54.835064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fb3b6b67-5647-4640-9022-a74959993d19 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1242 microversion: 2.1 time: 0.224975 Sep 19 16:13:54.847367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 5/10] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:54 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1242 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:54.906242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-249993149", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3002445e-0f83-4b0b-b1fa-f136b28412de"}], "security_groups": [{"name": "tempest-securitygroup--1862282357"}], "key_name": "tempest-keypair-132643020", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:55.511386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] validate_networks() for [('3002445e-0f83-4b0b-b1fa-f136b28412de', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:56.494117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:13:56.494915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:13:56.495810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:13:56.503503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:13:56.504598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:13:56.569549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:56.570903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:56.572558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:56.578792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:56.591649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:56.593100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:56.696964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:56.719518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for user 8cd9c727c0b444cb83497ea4f78e24d0 and project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:56.787801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:13:56.793916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:13:56.985222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-603cf2fb-60ca-4737-a5b4-5aa8b3eb4bec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 4.630339 Sep 19 16:13:56.986470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 6/11] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:13:52 2022] POST /compute/v2.1/servers => generated 383 bytes in 4633 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 19 16:13:57.013050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:57.014975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:57.038949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.041169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.046423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.048129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.051580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.057408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.256830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:13:57.259160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:13:57.260294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:13:57.262010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:13:57.263341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:13:57.304146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.306228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.307986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.309821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.313427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.318775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.407164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:13:57.424312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for user 754c1b5cb47746cf93c52d6dc0b8fa01 and project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:13:57.472790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:13:57.478646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:13:57.622603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4b1572b7-aacf-48ce-b678-cca7637c6b02 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.764965 Sep 19 16:13:57.641622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 6/12] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:13:54 2022] POST /compute/v2.1/servers => generated 402 bytes in 2793 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:13:57.655136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:57.659773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:57.694394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.694890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.715319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.716293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:57.716787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:57.719997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:57.760248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:57.842224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3653ba13-a71b-48d2-8074-4e8576c5e505 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1274 microversion: 2.1 time: 0.850231 Sep 19 16:13:57.847438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 7/13] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:56 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1274 bytes in 857 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:57.854634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aaf9f143-0332-4de1-9a6d-1d057d8d3eaa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:57.855259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aaf9f143-0332-4de1-9a6d-1d057d8d3eaa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:58.048132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:58.085254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-46a4b358-4560-470a-94f6-8731c9fa7320 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1246 microversion: 2.1 time: 0.441212 Sep 19 16:13:58.086912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 7/14] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:57 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1246 bytes in 445 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:58.121485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bbf9eb2d-111c-48cb-94d3-eae0b5abd926 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:58.122930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bbf9eb2d-111c-48cb-94d3-eae0b5abd926 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:58.167979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aaf9f143-0332-4de1-9a6d-1d057d8d3eaa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:58.189614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aaf9f143-0332-4de1-9a6d-1d057d8d3eaa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1220 microversion: 2.1 time: 0.338724 Sep 19 16:13:58.192815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 8/15] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:57 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1220 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:58.577565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bbf9eb2d-111c-48cb-94d3-eae0b5abd926 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:58.601389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bbf9eb2d-111c-48cb-94d3-eae0b5abd926 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1236 microversion: 2.1 time: 0.512288 Sep 19 16:13:58.604963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 8/16] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:58 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1236 bytes in 517 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:58.877342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:58.878456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:58.896210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:58.897595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:58.899295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:58.901164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:58.901862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:58.902715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.070037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:59.093484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f92bca58-9b64-4937-8ce5-8f4c565c6616 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1284 microversion: 2.1 time: 0.222769 Sep 19 16:13:59.095378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 9/17] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:58 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1284 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:59.120744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:59.122431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:59.151700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.152496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.158440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.161578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.163521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.164402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.217327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1758126765", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "668f712a-f576-4558-aa5d-ef858ef28f5d"}], "security_groups": [{"name": "tempest-securitygroup--1000621907"}], "key_name": "tempest-keypair-1741274330", "user_data": "}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:13:59.305080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:59.329420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7b35e52c-e04a-49f0-a4f2-cd9bccf3b344 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1238 microversion: 2.1 time: 0.218363 Sep 19 16:13:59.331584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 9/18] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:59 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1238 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:59.342313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:59.373452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:59.384159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.385288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.385942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.386960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.387637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.388223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.479694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] validate_networks() for [('668f712a-f576-4558-aa5d-ef858ef28f5d', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:13:59.533218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:59.548424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1f13b405-3daa-4460-8151-2d027fe80b56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1216 microversion: 2.1 time: 0.214807 Sep 19 16:13:59.549013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 10/19] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:59 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1216 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:13:59.654541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:13:59.656664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:13:59.669342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.669841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.670833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.671822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:13:59.672300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:13:59.672982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:13:59.819748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:13:59.850711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d6a7e96c-1e5b-4ede-bb93-5014fc0e90c0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1240 microversion: 2.1 time: 0.200671 Sep 19 16:13:59.851302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 11/20] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:13:59 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1240 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:00.138869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:00.139974 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:00.163788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.164728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.165773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.167378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.168430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.169394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.337054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:00.363927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c9324908-88f2-41fe-b3c6-55554a1b35f7 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1284 microversion: 2.1 time: 0.232910 Sep 19 16:14:00.365356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 12/21] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:00 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1284 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:00.405483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:00.406826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:00.441657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.442158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.442637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.443370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.443826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.448296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.471144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:14:00.472651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:14:00.473487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:14:00.474328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:14:00.475010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:14:00.517093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.518443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.519688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.521493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.522667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.523869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.576659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:14:00.591600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for user bd190d8e252342ca9232f7a0c966c715 and project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:14:00.606957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:14:00.609527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:14:00.640234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:00.664917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-084123b3-99b0-4635-8e62-8e1bc06ecbc2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1238 microversion: 2.1 time: 0.295392 Sep 19 16:14:00.665981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 13/22] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:00 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1238 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:00.693031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:00.694137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:00.707681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc7af0d7-5a2a-44c7-8e12-f1d4e2e2eb12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.499885 Sep 19 16:14:00.709237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 10/23] 10.222.0.57 () {62 vars in 1188 bytes} [Mon Sep 19 16:13:59 2022] POST /compute/v2.1/servers => generated 402 bytes in 1504 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 19 16:14:00.730115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:00.731101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:00.760210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.761014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.761843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.764976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.771941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.778338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.826843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:00.835647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:00.839819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:00.841680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d8b1ef88-aa2c-463f-b0e0-2662d5332515 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1282 microversion: 2.20 time: 0.124138 Sep 19 16:14:00.850967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 11/24] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1282 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:00.884290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:00.885339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:00.916734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.920505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.925293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.932165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:00.933906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:00.942779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:00.998958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:01.011302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e565da00-cec3-42d9-ba18-640005820040 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1216 microversion: 2.1 time: 0.342668 Sep 19 16:14:01.028211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 14/25] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:00 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1216 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:01.094006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:01.105763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4e1a3ec9-7c5b-4114-88cd-3046532debe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1240 microversion: 2.1 time: 0.227887 Sep 19 16:14:01.116964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 12/26] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:00 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1240 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:01.396946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:01.397904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:01.434887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:01.435938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:01.438791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:01.440175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:01.448324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:01.449723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:01.661801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:01.678278 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-14ebff11-2156-4e22-a72c-fbf55f67ffbe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1272 microversion: 2.1 time: 0.287120 Sep 19 16:14:01.679702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 15/27] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:01 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1272 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:01.710752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:01.720417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:01.752509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:01.753625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:01.754415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:01.763224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:01.764402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:01.765386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:01.872483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:01.875288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:02.086944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.129172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.136951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.147904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1f3b363f-a5d8-4b7c-8c69-cd3f682f450c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1282 microversion: 2.20 time: 0.279752 Sep 19 16:14:02.163855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 16/28] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1282 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:14:02.185259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:02.200613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:02.214498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.215748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.216768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.219835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.221141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.222445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.364445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.402337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8bd59f2e-4d56-4ff6-a669-14c4af84cd0b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1238 microversion: 2.1 time: 0.692867 Sep 19 16:14:02.407117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 13/29] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:01 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1238 bytes in 712 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:02.425722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:02.454108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:02.475302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=89,nova_api:INSERT=11 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:02.536250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.540590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.541925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.544164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.546007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.547206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.590290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.606120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4b478ee2-a55f-49ea-a938-97b58897936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1272 microversion: 2.1 time: 0.448729 Sep 19 16:14:02.608086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 17/30] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:02 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1272 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:02.721887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:02.722894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:02.729004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.729850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.730608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.731563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:02.732287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:02.733615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:02.839012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:02.850644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aeb879bb-d246-4ac7-bc4c-310aaf2e6558 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1272 microversion: 2.1 time: 0.132604 Sep 19 16:14:02.856391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 18/31] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:02 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1272 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:03.122692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:03.135396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9cac927e-38d8-45fc-9073-a79e90bb13cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1232 microversion: 2.1 time: 0.725714 Sep 19 16:14:03.136954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 14/32] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:02 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1232 bytes in 729 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:03.184722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:03.185759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:03.194325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=90,nova_api:INSERT=10 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:03.213018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.215842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.218292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:03.220274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.222180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.223100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:03.431314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:03.434008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:03.462266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.463782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.465687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:03.466991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.470311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.474575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:03.788555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:03.801324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6eca203e-07ae-4d0a-9183-b1dffc42fd52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1294 microversion: 2.1 time: 0.375749 Sep 19 16:14:03.801674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 15/33] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:03 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1294 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:03.825533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:03.827449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:03.854948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.856877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.860876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:03.864708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:03.869088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:03.869885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.013100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.041453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.053655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.059187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c1ef6e2a-0a99-451a-9c2f-358bf61585e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1278 microversion: 2.20 time: 0.881014 Sep 19 16:14:04.067443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 19/34] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1278 bytes in 890 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:04.095723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:04.107733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:04.115874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.121344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.131350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.132429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.132944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.133382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.185595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.202552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ac87872d-7a75-40d6-9a33-79c829a4bee8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1278 microversion: 2.1 time: 0.386977 Sep 19 16:14:04.204962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 16/35] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:03 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1278 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:04.235547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:04.237744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:04.271707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.276892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.281920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.285280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.291179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.296585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.318679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.336884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9a346829-1e7f-470f-8c0d-34c135aecfab tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1272 microversion: 2.1 time: 0.264991 Sep 19 16:14:04.337501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 20/36] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:04 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1272 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:04.830486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:04.831501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:04.845381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.846444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.847101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.848172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:04.848864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:04.856807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:04.877475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:04.897110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-214fd5cf-d1d0-4a1f-99c2-7b6565109318 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1232 microversion: 2.1 time: 0.690583 Sep 19 16:14:04.902866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 17/37] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:04 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1232 bytes in 697 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:05.109388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:05.110527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:05.158488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.163289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.164568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.165362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.166734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.167585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.168368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.169592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d341aabc-cf89-4c97-80cc-f1e097861f37 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1302 microversion: 2.1 time: 0.344224 Sep 19 16:14:05.179142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 21/38] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:04 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1302 bytes in 354 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:05.230953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:05.231570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:05.243642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.244153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.244673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.245405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.250288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.250680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.317984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.329278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.352617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.354277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-40670a4d-7f91-4e42-95eb-c10e5826572b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1278 microversion: 2.20 time: 0.261963 Sep 19 16:14:05.362159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 18/39] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1278 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:05.378288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:05.379259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:05.392540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.393828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.395196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.396894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.397972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.399911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.557809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.573636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ce1a2c2a-053e-432b-9f08-b8bf76a8e03d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1278 microversion: 2.1 time: 0.348275 Sep 19 16:14:05.590609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 22/40] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:05 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1278 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:05.696937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:05.708054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fbe9b7d3-a90a-414f-a5cb-fc6ce1bfe125 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.344690 Sep 19 16:14:05.709236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 19/41] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:05 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 347 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:05.938563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:05.939381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:05.947125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.947881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.948474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:05.949376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:05.949822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:05.950424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.002728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:06.150405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.162774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c83fbef5-adcd-4ec9-aede-7f2b42e45d69 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1288 microversion: 2.1 time: 0.227931 Sep 19 16:14:06.163444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 23/42] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:05 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1288 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:06.200484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:06.201771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:06.225936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.226888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.235481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.238257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.243107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.243871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.395000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.400167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:06.400747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:06.427115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38ceeb60-fe54-47fa-8245-ac8e00435ead tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1312 microversion: 2.1 time: 0.226866 Sep 19 16:14:06.429296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.429517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 20/43] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:06 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1312 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:06.430164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.430948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.431838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.432421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.435735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.602803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:06.604004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:06.615385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.616137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.618811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.619780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.623026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.624356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.657845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.667771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.669216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.687020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2dc22c1c-8853-4da2-a3f2-79a3d1ce3a93 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1278 microversion: 2.20 time: 0.295598 Sep 19 16:14:06.693902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 24/44] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1278 bytes in 305 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:06.733200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:06.734497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:06.760309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.761509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.762305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.764097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:06.764858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:06.770362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:06.963510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:06.981249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0cff1388-50d1-4d1b-9e70-38feec85bd32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.384497 Sep 19 16:14:06.982666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 21/45] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:06 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 387 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:07.020847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:07.039565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-70517be7-c2af-4097-94ac-7885b6b1ec3c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.309620 Sep 19 16:14:07.055093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 25/46] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:06 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:07.207596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:07.208578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:07.217230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.217882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.218457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.219273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.219827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.220346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.298900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:07.330690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-90509e95-b3e5-4518-8411-1c5b159d4917 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1288 microversion: 2.1 time: 0.126718 Sep 19 16:14:07.336578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 22/47] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:07 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1288 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:07.454882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:07.456426 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:07.464678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:07.470981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.471959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.472836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.474245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.477035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.478082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.689326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:07.705282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-15c7679a-9954-4cd2-ad5b-9b1b59fc8487 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1300 microversion: 2.1 time: 0.254078 Sep 19 16:14:07.719607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 26/48] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:07 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1300 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:07.752181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:07.752849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:07.777902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.778774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.779471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.781307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:07.781813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:07.782256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:07.953306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:08.049793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:08.054086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:08.057604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:08.072220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:08.094036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-24a7f357-2a1e-4860-9454-c53b1a4dc1aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1278 microversion: 2.20 time: 0.339576 Sep 19 16:14:08.110012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 23/49] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1278 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:08.119305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:08.128401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:08.134164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:08.137529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:08.145555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:08.148729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:08.150068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:08.150982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:08.181507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:08.181507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:08.181507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:08.184823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:08.189730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:08.190824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:08.869694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:08.934814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-06df1a14-5f2d-4366-87fe-89567fce1625 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.804407 Sep 19 16:14:08.948295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:08.950650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 24/50] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:08 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 846 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:09.020045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5d63ff02-ca93-4d49-845f-0d1d2529d4f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 1.010146 Sep 19 16:14:09.021396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:09.022213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:09.047296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 27/51] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:08 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 1038 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:09.072882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.074442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:09.076279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:09.077254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.079782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.087417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.090201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.094607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.120742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.126662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.128078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.129645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.130940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.147579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.228814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:09.408203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:09.447241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a385ed9d-4aef-4ca6-b73f-6776419ef1f3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1288 microversion: 2.1 time: 0.485458 Sep 19 16:14:09.450159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 25/52] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:08 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1288 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:09.478273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:09.482972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:09.541795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.548237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.548965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.570831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:09.572125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:09.574851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:09.733582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:09.747894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b31cd6db-4727-4692-9edf-efc974fedb3e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1300 microversion: 2.1 time: 0.707395 Sep 19 16:14:09.795986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 28/53] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:09 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1300 bytes in 756 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:09.984338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:09.991623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:10.046754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:10.050765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:10.051877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:10.063483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:10.065324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:10.068189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:10.309593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:10.325466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:10.344204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:10.347454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-55a94f3b-90b1-4246-b7a5-72726fbb3638 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1334 microversion: 2.20 time: 0.894477 Sep 19 16:14:10.347666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 26/54] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:09 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1334 bytes in 896 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:10.379704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:10.380326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:10.435462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:10.435960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:10.436410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:10.437188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:10.437680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:10.438141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:10.542864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=6 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:10.837405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=30 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:11.210006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.239933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3f7761f8-1b75-4102-9224-ea76cfbc2fe4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.871771 Sep 19 16:14:11.241339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 27/55] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:10 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 893 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:11.257134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:11.258147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:11.276375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.277147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.277592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.278329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.278821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.279253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.433020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.448350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-881db8fe-fd55-47ba-a57e-39e69cab0bb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1294 microversion: 2.1 time: 0.205362 Sep 19 16:14:11.449155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 28/56] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:11 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1294 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:11.459470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:11.461595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:11.477484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.478792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.479853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.482095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.483284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.484277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.683352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.700062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.700950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a93101e0-9f93-4cf4-9b3e-c9ff48b5ca60 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1300 microversion: 2.1 time: 0.248749 Sep 19 16:14:11.714411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 29/57] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:11 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1300 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:11.714832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:11.715899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:11.731929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.733266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.734352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.735792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:11.736651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:11.737479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-08816fb6-3ffb-4d06-91bf-ba3d84a1ac28 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 1.762641 Sep 19 16:14:11.737597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:11.761025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 29/58] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:09 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 1785 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:11.956563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.969307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.977381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:11.978755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-820e9380-e377-400e-aaed-58b11f26fe53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1342 microversion: 2.20 time: 0.274520 Sep 19 16:14:11.994985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 30/59] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:11 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1342 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:12.277020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:12.277918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:12.289801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=28 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:12.306299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.306837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.315173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:12.316037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.316526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.322027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:12.501250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:12.502859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:12.523769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.524757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.525388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:12.529238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.530260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.535262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:12.823320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:12.832541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c4700159-802e-4bae-8546-0fb3081513cd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1294 microversion: 2.1 time: 0.338163 Sep 19 16:14:12.839393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 31/60] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:12 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1294 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:12.848954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:12.855097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:12.865494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.869766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.870562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:12.871528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:12.879652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:12.880118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.059851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.068418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-77d05352-51ad-479e-96a0-7fe01bc1376d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1300 microversion: 2.1 time: 0.227633 Sep 19 16:14:13.069007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 32/61] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:12 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1300 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:13.074366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:13.075217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:13.083222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.083964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.084957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.085921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.087328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.087824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.158339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.171041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f143a820-f787-4264-ae90-d2273d351498 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.900972 Sep 19 16:14:13.172291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 30/62] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:12 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 903 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:13.178102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:13.179755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:13.192904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.194169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.195043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.196020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.196729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.197363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.366065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.391090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.392798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.393959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-98e31466-ec6f-4d32-afef-2671541f96ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1342 microversion: 2.20 time: 0.220482 Sep 19 16:14:13.394557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 31/63] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:13 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1342 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:13.500649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:13.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a297808-4b7a-4b78-bb08-1f7979b14019 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.464200 Sep 19 16:14:13.539721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 33/64] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:13 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:13.689549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "7163ee7c-30ec-4aff-b4c6-5fe173a3efe2", "tag": "4c4ec08a-9a03-4e16-91a0-f12747ea649c"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:13.721136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.721983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.722764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.771312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Creating event network-changed:4c4ec08a-9a03-4e16-91a0-f12747ea649c for instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 on hv-cinder-85837 Sep 19 16:14:13.772220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120663) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 19 16:14:13.788273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-2dd127e5-7f72-45ce-9fb7-c0159471880f req-29afdcd7-974b-465f-81dd-7df62f8dcf57 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120663) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 19 16:14:13.791414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.793760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.795423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.798506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.800029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.801245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.836258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.compute.rpcapi [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 19 16:14:13.839692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] Releasing lock "compute-rpcapi-router" {{(pid=120663) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 19 16:14:13.874343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-53ba6846-a216-4a33-8055-9cd6f1d08547 None None] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.477252 Sep 19 16:14:13.879269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 32/65] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:13.882223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:13.882881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:13.896226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.896701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.897315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:13.898687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:13.900201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:13.900814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.102285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:14.105894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:14.143353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.144287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.145093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.164031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.164707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.165195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.190847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:14.240552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c3c1abae-3816-46b6-a073-9b59ae09efa5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1294 microversion: 2.1 time: 0.361841 Sep 19 16:14:14.248550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 34/66] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:13 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1294 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:14.456059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0d7d2812-f954-45c7-9916-4bd9fcb96bcc", "tag": "88a993b9-866a-428d-a980-4b36a142d1f5"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:14.524275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.525009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.525596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.593102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Creating event network-changed:88a993b9-866a-428d-a980-4b36a142d1f5 for instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc on hv-cinder-85837 Sep 19 16:14:14.594122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120664) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 19 16:14:14.594122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e18868ff-5d43-4328-9b5e-b1683ed4f373 req-7e80bbfd-8d00-4494-9e47-5c3bc9ca5656 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120664) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 19 16:14:14.595233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.596169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.596855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.598267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.598912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.600180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.622901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.rpcapi [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 19 16:14:14.624433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] Releasing lock "compute-rpcapi-router" {{(pid=120664) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 19 16:14:14.687672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b4d3cf2d-0935-40e6-a2ae-6780595a0a53 None None] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.424605 Sep 19 16:14:14.688717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 35/67] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:14 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:14.709481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:14.719341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:14.745886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.746403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.746870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.747587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.748051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.748487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.833703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:14.884472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-64be69e5-4caf-4b7b-b7e7-06b3a5f646df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.790940 Sep 19 16:14:14.904280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 33/68] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:14 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 813 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:14.918750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:14.920876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:14.928702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.930074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.930811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:14.946749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:14.947701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.016s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:14.948387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:15.862735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:15.875233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-543f3960-71a4-45d5-899a-46d20694767a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 1.183421 Sep 19 16:14:15.879020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 36/69] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:14 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 1188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:15.898559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:15.899614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:15.906574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:15.909161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:15.909785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:15.911195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:15.922108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:15.923617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:15.953298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:15.999582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:16.020405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:16.027606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9d66bb88-ca88-4b81-a39a-8a809ca37641 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1352 microversion: 2.20 time: 1.121433 Sep 19 16:14:16.033677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 34/70] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1352 bytes in 1128 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:16.053585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:16.056462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:16.074213 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:16.076187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:16.078055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:16.080296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:16.081677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:16.087182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:16.675706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:16.686415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0df3ff68-df1e-41e1-8de4-481391d03ffe tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.806033 Sep 19 16:14:16.703948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 37/71] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:15 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 824 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:16.716395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:16.720390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:16.738023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:16.738887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:16.739609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:16.740582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:16.741229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:16.742404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:17.822792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:17.863243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-295e9022-e6c5-415e-b227-952b3707d71f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 1.156949 Sep 19 16:14:17.865982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 38/72] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:16 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 1161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:17.888902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:17.890839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:17.912844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:17.914594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:17.915924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:17.917660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:17.919896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:17.921293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:17.990665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:18.009216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6efee81f-426a-468c-bfec-5e3409ab04ec tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 1.973358 Sep 19 16:14:18.010640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 35/73] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:16 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 1976 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:18.026750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:18.027759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:18.034282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.043218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.043987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.046028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.046919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.047576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.260106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:18.271705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:18.289917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:18.292887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1bd42d3d-c403-47c0-a87e-a5945ab73ece tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1340 microversion: 2.20 time: 0.279944 Sep 19 16:14:18.308079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 36/74] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1340 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:18.327270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c5d17c4e-bcff-42fa-95c7-f7df3fb97a42", "tag": "a3ce130c-3698-4fbe-a5ac-58db4bc770fe"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:18.369696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.387170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.388084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.429924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] Creating event network-changed:a3ce130c-3698-4fbe-a5ac-58db4bc770fe for instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 on hv-cinder-85837 Sep 19 16:14:18.451858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-31738f00-2037-4572-b418-d4d302aa9275 req-7d1e79ff-85a5-42f7-a76f-32b22dce8e25 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.151633 Sep 19 16:14:18.453740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 37/75] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:18.477830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:18.479549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:18.507959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.509260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.512706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.514592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.515742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.516932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.654457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:18.707501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7e7cd811-e215-4173-a3a7-77fcc212cc3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.836645 Sep 19 16:14:18.708677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 39/76] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:17 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 841 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:18.752410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bae51cb6-fc0c-4343-9cd7-c04ef65d6b93", "tag": "84b6192c-6cd6-4692-a397-69b77dc536c9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:18.793023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.794084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.794957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.815564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] Creating event network-changed:84b6192c-6cd6-4692-a397-69b77dc536c9 for instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 on hv-cinder-85837 Sep 19 16:14:18.826744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-b2511bab-9cb4-4ffb-9f00-b96be364fee8 req-80c067e8-d3e2-4115-9da0-f298c6298696 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.111488 Sep 19 16:14:18.830328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 40/77] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:18.919221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:18.920370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:18.929267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.929790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.930306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:18.931098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:18.931782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:18.932252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.308541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.321543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d268100f-6888-40df-8e6f-01389bae0fd4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.408355 Sep 19 16:14:19.323167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 41/78] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:18 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:19.328936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:19.331704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:19.340732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.341924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.342641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.343600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.344348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.345063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.363693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.394036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4bf80358-4f77-479d-af4e-5a8725210752 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.937257 Sep 19 16:14:19.403490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 38/79] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:18 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 949 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:19.427657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:19.432066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:19.454588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.461697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.462592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.468549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.472059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.473006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.716469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.736569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9c21527-1b7c-4b5c-b13d-81fb560e21a0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.411567 Sep 19 16:14:19.738837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 42/80] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:19 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:19.749832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:19.752416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:19.764720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.765450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.766203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.793942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:19.794887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.027s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:19.797140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:19.806028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:19.936577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.947299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.949032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:19.950334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0db87cbe-5698-4bd8-944a-d986efc1fe98 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.544765 Sep 19 16:14:19.951513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 39/81] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 547 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:20.302560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:20.321573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a4a5292c-3bc4-4446-b5f1-8812d9e9dc29 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.580189 Sep 19 16:14:20.339721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 43/82] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:19 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 600 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:20.378961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:20.380004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:20.392915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.393949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.394789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.396060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.396823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.397619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.451254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:20.473325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:20.500521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.501336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.506090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.509241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.514837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.515678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.661096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:20.680217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fdff480e-8a07-40e5-a8cc-6a95b2e1a5df tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.308856 Sep 19 16:14:20.680975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 40/83] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:20 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:20.686487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d5cfe70-1a06-4c64-91bf-4681779bd260", "tag": "bf337196-4336-4bb9-9fcf-4dfbcb7910a6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:20.729935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.730753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.731313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.747198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:20.766536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] Creating event network-changed:bf337196-4336-4bb9-9fcf-4dfbcb7910a6 for instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 on hv-cinder-85837 Sep 19 16:14:20.773495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-9523c56a-20c2-486f-ad79-6444b188c5e5 req-4079ab13-f516-4aab-96d9-0d48ebdbafb4 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.091082 Sep 19 16:14:20.776471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 41/84] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:20.791328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:20.792378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:20.811183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.811781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.812311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.813117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:20.813651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:20.814242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:20.925376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:20.941631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dd10acc9-5133-4893-9644-417185b05bb4 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.495594 Sep 19 16:14:20.976935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 44/85] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:20 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:20.998099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:20.999685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:21.017063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.017978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.018692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.019671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.020384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.021627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.202635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:21.215643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-28eee4f3-d465-49fa-964d-24b4b136b8af tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.430556 Sep 19 16:14:21.216248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 42/86] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:20 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:21.341905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:21.354080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:21.361147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:21.362437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:21.373101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:21.376539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c7fb9ec8-6eec-4948-8695-a82467669099 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.397982 Sep 19 16:14:21.378764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.381095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 45/87] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:21.381498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.383114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.384236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.385402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.385904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.712609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:21.713698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:21.743465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.744319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.745079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.746335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:21.760458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:21.787733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.041s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:21.798557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:21.805771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-df135b41-54b9-4488-9191-9cd01399a0e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.429751 Sep 19 16:14:21.839961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 43/88] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:21 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:21.990657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:21.991578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:22.007613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:22.011351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:22.012873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:22.024238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:22.024952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:22.025409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:22.720392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:22.732854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4cfbd8e4-ebcc-4451-a25f-39697e6dd847 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 1.028694 Sep 19 16:14:22.735364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 46/89] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:21 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 1032 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:22.752809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:22.757230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:22.778667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:22.779243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:22.780429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:22.781442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:22.784756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:22.785508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:22.969496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:22.983272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e56b837c-7d76-4942-bd52-258d0f16b49e tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.996466 Sep 19 16:14:22.998418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 44/90] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:21 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 998 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:22.998418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:22.998418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:22.999161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:22.999695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.001873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.003004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:23.003753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.004418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.208772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:23.235228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-03025537-f356-4814-9ff5-ad1f4c6b7ae0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.493440 Sep 19 16:14:23.239678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 47/91] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:22 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:23.253465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:23.254065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:23.273263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:23.273443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.273827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.274585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:23.275047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.275487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.663255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:23.700396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:23.708130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:23.715081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-247eeb5a-ac5f-4135-be0f-6fd0f0ced5aa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.726053 Sep 19 16:14:23.741389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 45/92] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 748 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:23.787645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:23.789929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:23.802519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:23.803393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.804409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.805669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:23.806778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:23.807724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:23.840644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:23.852622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:23.865183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3fa16b13-bd8c-48b0-903c-d813fdedb400 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.620260 Sep 19 16:14:23.869826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 48/93] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:23 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 628 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:24.028867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:24.029768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:24.040920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.042158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.042912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.047776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.049089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.050797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.150343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:24.160901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4ceae0fb-7cd3-4a61-ac60-a161372f58f0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.383860 Sep 19 16:14:24.165166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 46/94] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:23 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:24.274326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:24.278776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:24.322284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.323665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.324769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.326307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.327441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.329621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.363385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:24.373424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ac16bb21-5af1-4ecc-b015-c9cfa7ab0916 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.349276 Sep 19 16:14:24.393769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 49/95] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:24 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:24.439719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:24.457924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.461084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.461535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.492341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] Creating event network-changed:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:14:24.502557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-d8127a2b-3108-4b65-82eb-7cdb14c06a00 req-f8e9dfba-2abe-4dfa-b3ef-0bf55b53a04e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.067060 Sep 19 16:14:24.506266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 50/96] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:24.783905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:24.785518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:24.806094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:24.821444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.834889 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.836176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:24.838210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:24.839648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:24.840691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.178576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.200479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.207412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.213305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a1be7823-1e3a-420d-b2f9-6f7812cdb7bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.442043 Sep 19 16:14:25.216072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 51/97] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:24 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:25.229759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:25.231708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:25.247156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.248692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.249678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.251440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.252399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.253364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.515519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.530737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b6867045-0f17-4b9c-94f3-c1ecfa6a2290 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 1.267368 Sep 19 16:14:25.532777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 47/98] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:24 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 1271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:25.541331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:25.542890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:25.554321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.555532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.556639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.557850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.558649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.559356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.612938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.629784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d6d651df-2d09-440c-837a-2aa33add8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.411106 Sep 19 16:14:25.639219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 52/99] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:25 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:25.648365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:25.649307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:25.667036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.671446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.672195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.673541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:25.674331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:25.675316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:25.925635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:25.953242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-08c209e4-0c26-4752-8296-786abdb8f173 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.418170 Sep 19 16:14:25.954172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 48/100] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:25 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:26.075834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:26.103790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-53c7bb42-7045-4e1c-a593-2f0317e370c5 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.462283 Sep 19 16:14:26.105508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 53/101] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:25 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:26.263480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:26.266958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:26.290873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:26.291767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:26.295267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:26.297228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:26.297977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:26.300294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:26.568637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:26.572306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:26.595969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:26.596841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:26.597606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:26.605783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:26.606822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:26.607593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:27.302875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:27.332182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f0394e16-d599-413f-a5b9-ee781c7d2071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.771316 Sep 19 16:14:27.343701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 54/102] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:26 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:27.353449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:27.368185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:27.388699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:27.389702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:27.400231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:27.401518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:27.402706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:27.407583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:27.777693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:27.810712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:27.816298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:27.822577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f02df0bb-eabb-46b6-97dd-9a4e890e6ab1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 1.564618 Sep 19 16:14:27.825181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 49/103] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:26 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 1572 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:27.843432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:27.845532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:27.858601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:27.859976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:27.862021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:27.864181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:27.866759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:27.870728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:28.260986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:28.273912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3883ed8a-5a60-43ce-b2d1-f7fd1026a9da tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.928213 Sep 19 16:14:28.279876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 55/104] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:27 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 935 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:28.315306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:28.316450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:28.336988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:28.337795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:28.338544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:28.339811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:28.340577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:28.341284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:28.845118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:28.871573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5bc4885e-c00a-4a0e-9119-d83d045efd0a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.588943 Sep 19 16:14:28.875795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 56/105] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:28 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 595 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:28.897149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:28.898145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:28.914435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:28.920227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:28.921004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:28.930669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:28.931446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:28.932056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:28.996636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.018484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-35ffad47-8c9d-47ae-9728-63a72690a66f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 1.190732 Sep 19 16:14:29.023288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 50/106] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:27 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 1198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:29.040509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:29.044267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:29.071347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.077582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.081505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:29.085380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.087270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.087518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:29.458061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.476141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.480122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.481582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9bb4f215-2d9e-4428-a526-1b2b452f8d6b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.457063 Sep 19 16:14:29.482321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 51/107] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:29.517343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:29.521036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:29.539175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.540300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.540983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:29.545485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.546036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.546511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:29.592586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.639336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ef6273b1-db69-4373-9883-14a2c3fbc751 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.761645 Sep 19 16:14:29.640928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 57/108] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:28 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 764 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:29.796055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:29.804600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7befdf89-0111-470f-8372-f2c94b8cb601 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.320997 Sep 19 16:14:29.805197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 52/109] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:29 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:29.906007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:29.915922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:29.941484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.942848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.944041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:29.945676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:29.946870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:29.947704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.056221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:30.057519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:30.072317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.073395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.074003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.075335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.076019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.079239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.417509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:30.426828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bec5e043-a907-4186-bf06-311f257aa095 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.533127 Sep 19 16:14:30.427753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 58/110] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:29 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:30.526881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:30.528052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:30.535944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.536881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.537556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.540116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.540915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.541572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.735493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:30.768766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-98d81495-76e0-4f29-8a68-0c2e7aa0010c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.723367 Sep 19 16:14:30.770206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 53/111] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:30 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 726 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:30.789892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:30.794680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:30.810269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.811563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.812607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.814785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:30.816044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:30.817082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:30.960904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:30.980502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:30.984019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:31.008624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bd32e600-b7a1-4cb1-af97-b27a0f490178 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.463396 Sep 19 16:14:31.009779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 59/112] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:31.205167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:31.217823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f6d7de47-8803-42d2-aa02-a5f70df9a8a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.445382 Sep 19 16:14:31.219051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 54/113] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:30 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 448 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:31.240639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:31.241983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:31.261833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.262976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.263566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.264572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.265123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.265771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.380734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7163ee7c-30ec-4aff-b4c6-5fe173a3efe2", "name": "network-vif-plugged", "status": "completed", "tag": "4c4ec08a-9a03-4e16-91a0-f12747ea649c"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:31.400857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.401558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.402176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.417665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] Creating event network-vif-plugged:4c4ec08a-9a03-4e16-91a0-f12747ea649c for instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 on hv-cinder-85837 Sep 19 16:14:31.424037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-6a62076b-2642-4c33-9f3d-02c6f99a0a2f req-e7cacf96-4620-4f0f-b099-e5bf48674f78 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.411310 Sep 19 16:14:31.425020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 60/114] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 414 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:31.454828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:31.455751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:31.463231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.463973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.464597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.465466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.466112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.466577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.471618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:31.531301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:31.541555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-af7bee99-fce4-4301-abd9-d10f9c12dc5f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.320793 Sep 19 16:14:31.542243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 55/115] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:31 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:31.654765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:31.669884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8cce5853-de52-4421-9704-1b33ef00b0d1 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.219223 Sep 19 16:14:31.681917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 61/116] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:31 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:31.813444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:31.814507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:31.823030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.823666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.824279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:31.825160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:31.825732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:31.826252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.010162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.019180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ab54c985-5b20-4254-b52d-b649f91d9b4c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.212456 Sep 19 16:14:32.019855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 56/117] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:31 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:32.048439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:32.049551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:32.060375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.061137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.061737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.063022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.063947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.064827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.286943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:32.294943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:32.313698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.319702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.320607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.321403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.323110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.323325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.324408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.324748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.325655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.325888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-455f20a8-27d5-44f2-9639-63d53f70b571 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.283144 Sep 19 16:14:32.338411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 62/118] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:32.594406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.594406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2f0ca601-4d83-4f22-ae99-903ddaacb66b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.334134 Sep 19 16:14:32.594406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 57/119] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:32 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:32.622361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:32.623669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:32.634820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=97 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:32.642613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.644922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.645696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.648204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.649741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.650846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.705030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:32.706012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:32.724869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.725673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.726353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.728027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:32.729121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:32.730297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:32.780461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:32.973137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:32.984652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a4040b05-044f-413c-ab2a-5c8b61426943 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.391135 Sep 19 16:14:32.990540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 63/120] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:32 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:33.068458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:33.069590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:33.080464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.088241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.089659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e4fdee3-dd3e-4a26-a910-7de6a49e84e6 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.392122 Sep 19 16:14:33.090732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.094245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.096750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.097846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.099606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.103088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 58/121] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:32 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:33.283300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.292734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5d3c8da9-a5f0-4f50-8219-87d799d8268b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.241686 Sep 19 16:14:33.293485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 64/122] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:33 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:33.379066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:33.379843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:33.381781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=79 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:33.388448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.388992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.389646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.390486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.390951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.391384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.444284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5d17c4e-bcff-42fa-95c7-f7df3fb97a42", "name": "network-vif-plugged", "status": "completed", "tag": "a3ce130c-3698-4fbe-a5ac-58db4bc770fe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:33.455658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.459487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.459487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.471403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] Creating event network-vif-plugged:a3ce130c-3698-4fbe-a5ac-58db4bc770fe for instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 on hv-cinder-85837 Sep 19 16:14:33.477623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-b46b4125-5689-47f5-99d6-4cced20da76f req-2dd4e2e4-d8ad-484d-96a8-7208f2903c43 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038551 Sep 19 16:14:33.478237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 65/123] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:33.591288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.603658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.605140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.606302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-29d6a94d-f823-411d-8568-c7eea897a05f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.231503 Sep 19 16:14:33.606919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 59/124] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:33.637474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:33.638291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:33.649355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.649489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.650108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.651298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:33.651988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:33.652884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:33.866351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:33.879414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2a377d64-3eea-4395-aa12-3275124b73ef tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.249490 Sep 19 16:14:33.880370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 66/125] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:33 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:34.027308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:34.027951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:34.034751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.035649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.036261 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.037187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.037840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.038265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.155254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:34.158589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:34.177653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.178975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.181520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.182548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.183470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.184588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.231296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:34.242164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2873b64-b80c-4a2f-b71a-03c5d4a8a90e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.217570 Sep 19 16:14:34.255235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 60/126] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:34 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:34.320517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:34.321888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:34.331683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.333168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.333847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.335539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.336499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.337143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.422315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:34.433057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f96a6ce5-db79-496c-8d4e-5c18b16de8cc tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.319384 Sep 19 16:14:34.435932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 67/127] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:34 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:34.643691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:34.650119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:34.690810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:34.692974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.698179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.700113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.703989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.704486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.708949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.722873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-136e8930-b05e-44b9-b8fb-b358985eedbd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.406264 Sep 19 16:14:34.726980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 61/128] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:34 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:34.926480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:34.938815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:34.992548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.993537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.994200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:34.995841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:34.997243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:34.999725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.253365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:35.276580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:35.281451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:35.286869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-145bd648-044a-4d0d-9554-c7865ea9a99e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.651125 Sep 19 16:14:35.290905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 68/129] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:35.314466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:35.316062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:35.328271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.336390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.337750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.340040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.341124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.342047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.449321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:35.472138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a01dc746-a3e6-4dc6-bd5d-cd1deafc5f30 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.566409 Sep 19 16:14:35.474231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 62/130] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:34 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:35.489470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:35.492243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:35.504806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.506009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.508049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.509586 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.511334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.512579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.816240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:35.834170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c58c3150-94fd-4a1d-8f38-db67a5e84170 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.357048 Sep 19 16:14:35.835781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 63/131] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:35 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:35.846399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d7d2812-f954-45c7-9916-4bd9fcb96bcc", "name": "network-vif-plugged", "status": "completed", "tag": "88a993b9-866a-428d-a980-4b36a142d1f5"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:35.877732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.878113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.878776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.894857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] Creating event network-vif-plugged:88a993b9-866a-428d-a980-4b36a142d1f5 for instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc on hv-cinder-85837 Sep 19 16:14:35.902028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-64f7a993-0d0f-4f8c-bd27-096f5c01b134 req-40c8581e-7f1b-4231-9262-6ce9091bdc71 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064358 Sep 19 16:14:35.902848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 64/132] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:35.909842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:35.911240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:35.938253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.939052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.939511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:35.940236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:35.941852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:35.942661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.021382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.033552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-03b7a583-662a-453b-9edb-d96498b1a5d9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.738961 Sep 19 16:14:36.046293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 69/133] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:35 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 755 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:36.207577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.216664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-450771d6-2ffe-4ba8-84c0-9a1dd9792243 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.311617 Sep 19 16:14:36.216664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 65/134] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:35 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:36.341472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:36.342083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:36.348645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.349163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.349596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.350832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.351285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.351714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.490298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.500003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.501453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.501580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:36.502334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:36.502603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-54dfd1f6-2f1c-4bf7-b34a-5285ddd75681 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.165137 Sep 19 16:14:36.503169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 70/135] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:36.509552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.510041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.510511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.511216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.511657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.512074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.650086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:36.658357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ce52b2b6-8c53-4a37-8503-af3e94b84039 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.161001 Sep 19 16:14:36.658981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 66/136] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:36 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:36.885478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:36.886143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:36.892403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.892881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.893305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:36.893992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:36.894461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:36.894877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.045339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.057329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:37.058017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:37.058162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1f705ffa-8eca-43bb-a1c9-c7c35d1dec01 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.176020 Sep 19 16:14:37.059135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 71/137] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:36 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:37.064568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.065063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.065512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.066217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.066714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.067144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.226670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.240556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-24976f32-66e4-4c81-a0f9-712d8e8248ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.186874 Sep 19 16:14:37.241413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 67/138] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:37 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:37.241845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:37.242642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:37.248904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.249386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.249816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.250546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.250997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.251423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.414670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.423229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0e48a1bb-da34-49a4-af5a-4d2f071b074a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.185744 Sep 19 16:14:37.423844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 72/139] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:37 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:37.528746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:37.529268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:37.535477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.535963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.536399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.537098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.538166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.538627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.674667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.683374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:37.683645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.684301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:37.685095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.686230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c587e9c7-f3ea-4661-93b4-bf7d9c4762b0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.161605 Sep 19 16:14:37.686833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 68/140] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:37.692503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.693115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.694149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.695124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:37.695876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:37.696377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:37.877203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:37.886294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9373d40e-ee66-4de5-b844-4bd6d7d5cec7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.207075 Sep 19 16:14:37.887045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 73/141] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:37 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:38.098052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:38.098766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:38.105108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.105601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.106029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.106763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.107210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.107628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.277277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:38.277536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.278196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:38.286551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.286551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.286551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.287183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.288258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.288757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.295681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ba310986-e3e8-4197-b977-f0b64035268c tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.200932 Sep 19 16:14:38.296955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 69/142] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:38 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:38.438967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.447488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:38.447769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-479c4fd8-765b-4e8c-827d-3e5da3a86873 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.177833 Sep 19 16:14:38.448379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 74/143] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:38 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:38.448569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:38.455745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.456339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.456827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.457562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.458118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.458577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.599597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.608433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-49ce44dc-079f-48df-8498-e08eb677de39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.165675 Sep 19 16:14:38.609036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 70/144] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:38 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:38.713437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:38.714079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:38.720481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.720967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.721393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.722102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.722592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.723017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.915460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.915696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:38.918927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:38.925749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.927889 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:38.929593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d5444bf7-946c-4d7a-82eb-7de161cde466 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.221375 Sep 19 16:14:38.930423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 75/145] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:38.940892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.941806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.942635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:38.947086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:38.948603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:38.950482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.158259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bae51cb6-fc0c-4343-9cd7-c04ef65d6b93", "name": "network-vif-plugged", "status": "completed", "tag": "84b6192c-6cd6-4692-a397-69b77dc536c9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:39.178250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.191228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.192519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.215748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] Creating event network-vif-plugged:84b6192c-6cd6-4692-a397-69b77dc536c9 for instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 on hv-cinder-85837 Sep 19 16:14:39.224469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-891d1515-7709-4dd7-97c2-1525a0e2a030 req-c3284b26-0d4f-427a-8da6-da6dcb3063be service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073596 Sep 19 16:14:39.225762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 76/146] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:39 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:39.246579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:39.257512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aae457f5-1f3f-4661-99df-fbf49c9f8506 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.348833 Sep 19 16:14:39.258115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 71/147] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:38 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:39.350229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:39.354480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:39.385978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.385978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.385978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.385978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.386453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.386453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.480398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:39.482309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:39.501461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.502720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.504198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.506878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.508346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.509409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.561856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:39.574077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2a296169-14a8-4248-ba4f-b5153ce788ba tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.250062 Sep 19 16:14:39.575584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 77/148] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:39 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:39.642676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:39.644082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:39.653528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.654334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.655040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.656026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.656708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.657336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.774072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:39.789669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3dbb06e9-688c-4f7b-9af4-c240a0591631 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.321247 Sep 19 16:14:39.791465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 72/149] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:39 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:39.878936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:39.888385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4d03820e-8d71-44ae-9d52-4714561530ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.257701 Sep 19 16:14:39.888986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 78/150] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:39 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:39.957879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:39.959164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:39.967202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.967975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.968473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:39.969275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:39.969825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:39.970564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.142679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:40.153775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:40.155890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:40.157541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8cf4d83f-a718-4286-9b55-53f8aff49d96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.203746 Sep 19 16:14:40.158340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 73/151] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:40.282251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:40.282877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:40.289751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.290238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.290709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.291922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.292717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.293177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.298987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:40.512824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:40.525345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-41734a1d-3856-4d07-a38d-4f7a4e980b39 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.246975 Sep 19 16:14:40.526038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 79/152] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:40 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:40.604187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:40.605372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:40.616332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.617117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.617888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.619196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.620021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.620746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.799969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:40.819051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:40.820168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:40.829612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-111e9cea-fa1e-4b32-b70b-751a922b3358 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.230418 Sep 19 16:14:40.830737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 74/153] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:40 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:40.833810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.834668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.835123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.835922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.836450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.836950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.920759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:40.921888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:40.933615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.934513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.935374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:40.936665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:40.937481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:40.938250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.035793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.047731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1491717a-3129-4c10-8ba7-95c602ab7056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.235780 Sep 19 16:14:41.048439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 80/154] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:40 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:41.187048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:41.187870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:41.197018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.197700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.198351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.199353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.200007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.200539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.201008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.206548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:41.218246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5c5c51fa-8401-40d1-80b6-43b6fb3d4680 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.305211 Sep 19 16:14:41.225647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 75/155] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:40 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:41.285150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d5cfe70-1a06-4c64-91bf-4681779bd260", "name": "network-vif-plugged", "status": "completed", "tag": "bf337196-4336-4bb9-9fcf-4dfbcb7910a6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:41.303466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.304247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.305010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.327893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] Creating event network-vif-plugged:bf337196-4336-4bb9-9fcf-4dfbcb7910a6 for instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 on hv-cinder-85837 Sep 19 16:14:41.337252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-bc0d8506-c83d-455e-a523-aa30718ea348 req-29418b11-9a63-4c7a-8b98-656264bc3f92 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059299 Sep 19 16:14:41.338324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 76/156] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:41.447874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.486554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.507282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.510432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f5bbf83d-1050-4684-b496-83b7c0c6cf2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.327144 Sep 19 16:14:41.514664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 81/157] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:14:41.560175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:41.560809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:41.568046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.568854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.569353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.570082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.570611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.570995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.746922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:41.760718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8ba254bf-a317-41a5-9834-8d7f80c1eef3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.207392 Sep 19 16:14:41.761694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 77/158] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:41 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:41.872509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:41.873691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:41.884561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.885426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.886188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:41.887484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:41.888309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:41.889124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.088660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:42.089674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:42.096795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.097780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.098657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.101643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.102716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.103346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.109646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.126331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-603ebdff-cfa6-4889-aafa-290d93debe29 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.260544 Sep 19 16:14:42.127135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 82/159] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:41 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:42.253384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:42.254010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:42.273470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.274547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.277924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.278996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.283322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.290010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.474551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.485744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e5d63e13-7c0e-4207-acec-cb8f8266d688 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1320 microversion: 2.1 time: 0.398546 Sep 19 16:14:42.486505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 78/160] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:42 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1320 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:42.570686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:42.571316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:42.576278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.586157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ec58c37a-c716-4677-a3c2-766b1339d560 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.336619 Sep 19 16:14:42.586849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.587851 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.588401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 83/161] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:42 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:42.591645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.592793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.593545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.594314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.790683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:42.791049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:42.800018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.800511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.802250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.803318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:42.807651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:42.808695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:42.916925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.932021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.935196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:42.936624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1553b7e3-9856-4767-a5a5-512f71976bb3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.370686 Sep 19 16:14:42.937573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 79/162] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:42 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:43.161743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:43.163138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:43.181259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.182750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.184558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.185258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:43.187320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.188456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.190580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.214331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ac580330-36c0-44f5-a5ec-ed92df8907be tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.425272 Sep 19 16:14:43.215052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 84/163] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:42 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:43.410305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:43.421572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-18b63601-cb43-4e26-b5bf-eeef41419f0f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1314 microversion: 2.1 time: 0.265412 Sep 19 16:14:43.422186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 80/164] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:43 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1314 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:43.533249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:43.534428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:43.546075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.547062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.547835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.549109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.550024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.551043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.612665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:43.613786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:43.621704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.622461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.623060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.624303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.625022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.625559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.846043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:43.891688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6945b43a-28be-45c1-b31c-451ec3b0bc5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1514 microversion: 2.1 time: 0.363652 Sep 19 16:14:43.907824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 85/165] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:43 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1514 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:43.955292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:43.962477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:43.974301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.976154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.977205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:43.978772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:43.979896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:43.980901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.074885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:44.084578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-27227dc1-3582-4e07-9815-29f53eaf8418 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.476456 Sep 19 16:14:44.085850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 81/166] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:43 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:44.121317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:44.125160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:44.137828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'flavor' on Instance uuid 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:14:44.138250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.139490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.140924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.142565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.143728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.144919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.419618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1b909152-0e52-459d-82c3-daa1796973c7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2/action" status: 200 len: 32 microversion: 2.1 time: 0.480864 Sep 19 16:14:44.420702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 86/167] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:14:43 2022] POST /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2/action => generated 32 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 19 16:14:44.426452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:44.433537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:44.464313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.465507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.466673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.472385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.473533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.474752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.529727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:44.555774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:44.571659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:44.574300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4a2ddf8a-b6dc-4fee-b0d9-6e68c1ff0a3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.486699 Sep 19 16:14:44.575722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 82/168] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 489 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:44.585543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:44.586712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:44.612544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.623474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.624531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.626132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.627193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.628129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.925600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:44.942566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bd3a0e16-c867-4a79-a397-4751bf7f1032 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.520437 Sep 19 16:14:44.943684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 87/169] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:44 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:44.949866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:44.950537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:44.957659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.958339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.958923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:44.960208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:44.960664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:44.961096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.029614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.042578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-36706c84-fd65-4de4-8d33-7302e56e5e1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.462715 Sep 19 16:14:45.047951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 83/170] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:44 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:45.061071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "name": "network-vif-plugged", "status": "completed", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:45.075860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:45.076499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:45.076944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.127341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] Creating event network-vif-plugged:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:14:45.139021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-54009f55-2c0d-4ab5-b0a0-cb31b391b8e4 req-1371c9ee-114b-4df8-9cf0-13f7b568d382 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.082850 Sep 19 16:14:45.140149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 84/171] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:45.158608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:45.161050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:45.174819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:45.175942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:45.176821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.179330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.180218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:45.182963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:45.185560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.188134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0875d067-5d5b-4c57-8d73-0b200d198ea2 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1501 microversion: 2.1 time: 0.243351 Sep 19 16:14:45.189270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 88/172] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:44 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1501 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:45.446970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.461901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7cfed901-77ea-4727-8535-65b74a4f1550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.316629 Sep 19 16:14:45.462089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 85/173] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:45 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:45.629724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:45.632029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:45.650536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:45.655508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:45.656317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.657620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:45.658506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:45.659303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:45.929081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.942717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.948393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:45.949930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9e109c06-7f89-48a7-9d6a-0aba15523ce4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.324814 Sep 19 16:14:45.968402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 89/174] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:45.990733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:45.991390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:46.002224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.004306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.006617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.007675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.008852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.009730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.072685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:46.073307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:46.079900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.080782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.081436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.082192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.082883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.083553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.317582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:46.335098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a026943b-6053-45e1-84c2-d6aba6e1ef17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.348798 Sep 19 16:14:46.336160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 86/175] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:45 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:46.366331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:46.380272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a9b28b8-7ab9-477e-94f0-7c608b2e221d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.311617 Sep 19 16:14:46.381475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 90/176] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:46 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:46.508331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:46.509435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:46.531407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.532274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.533022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.534494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:46.535328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:46.536094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:46.854273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:46.869130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3e8eb464-a677-4b0c-b907-033a9ecc27ba tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1367 microversion: 2.1 time: 0.368504 Sep 19 16:14:46.871129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 87/177] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:46 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1367 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:46.985249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:46.986354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:47.000311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d9042ac7-23a7-4a1e-939e-3ddd6806c58e"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:47.002832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:47.021642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.022198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.022671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.023402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.023847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.033592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.034343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.035148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.035786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.040906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.041615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.042203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.396183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:47.421971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:47.425956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:47.429077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e34cfe97-8e5b-452d-a1aa-dbb4dc065957 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.446980 Sep 19 16:14:47.431300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 91/178] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:46 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:14:47.441059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:47.442576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:47.461289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.463490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.464586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.480836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.482649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.486060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.851839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:47.862300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c19f5f70-a1b9-4371-a7cf-88e7dbeddd23 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.429097 Sep 19 16:14:47.862939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 92/179] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:47 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 431 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:47.868606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:47.869665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:47.876795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.877614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.878241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:47.879464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:47.880015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:47.881492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.083244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:48.088698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:48.108374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b6d47620-fd75-4a8e-b5b1-705a2c7ad5d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.243829 Sep 19 16:14:48.110143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 93/180] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:47 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:48.119209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:48.120448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:48.134507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.135648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.137094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.139276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.140677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.141483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.388270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:48.399978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c25164e6-34f1-4664-86cb-b5cb95a69d22 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1556 microversion: 2.1 time: 0.287843 Sep 19 16:14:48.401067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 94/181] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:48 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1556 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:48.429422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:48.430230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:48.437408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.438150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.439218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.440372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.441138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.442119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.599055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5751c7a6-48e4-4ac1-9153-818fada4ebf5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.169009 Sep 19 16:14:48.599055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 95/182] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:14:48 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-interface => generated 284 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:48.627311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:48.636043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:48.652427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.654482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.656508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.658175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.659338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.660418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.966126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:48.976655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:48.978581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:48.980315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-13050bd5-58ee-4ef9-bcf2-6682cd9b923a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.375753 Sep 19 16:14:48.981013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 96/183] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 378 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:48.987703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:48.988567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:48.996044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.996764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.997382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:48.998227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:48.998856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:48.999420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:49.186208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:49.198432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-93a47593-14b8-4417-948e-f35e6bdd35d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1362 microversion: 2.1 time: 0.216070 Sep 19 16:14:49.199045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 97/184] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:48 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1362 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:49.203867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:49.206415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:49.211311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:49.211827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:49.212247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:49.212982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:49.213442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:49.213901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:49.381479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:49.390807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e2702c9-cd8e-4084-808c-20f01b5fbb06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.189919 Sep 19 16:14:49.390807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 98/185] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:49 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:50.029153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:50.035157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:50.060868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.061787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.062721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.064058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.069686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.070595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.276397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:50.289240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:50.291407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:50.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b9466416-2a01-415f-bf57-0697099546af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.272536 Sep 19 16:14:50.294088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 99/186] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:14:50.299903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:50.300808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:50.309179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.310086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.310742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.311662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.312307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.313142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.585863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:50.595471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1ab12261-a06f-438a-9bda-28a1721be22e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1548 microversion: 2.1 time: 0.300025 Sep 19 16:14:50.603289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 100/187] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:50 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1548 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:50.611059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:50.612494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:50.622349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.623305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.624859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.633139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.640403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.641126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.828764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2f5d223b-eb57-46e5-bde9-2c888da8593f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "POST /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.832186 Sep 19 16:14:50.830024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 88/188] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:14:46 2022] POST /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc/os-volume_attachments => generated 194 bytes in 3834 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:50.837552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:50.838530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:50.847935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.848810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.850301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.851439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:50.852147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:50.852842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:50.987361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:51.000549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-62aada77-9779-4b53-8472-fbd232130cb9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.395410 Sep 19 16:14:51.001747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 101/189] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:50 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:51.019688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-813304126", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "networks": [{"uuid": "67cc7387-dada-400e-a4de-f0effa2261f1"}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:51.097914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-74c04347-b31a-4a89-b169-633c4d6e15db tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.266018 Sep 19 16:14:51.098621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 89/190] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:14:50 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-interface => generated 284 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:51.116985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] validate_networks() for [('67cc7387-dada-400e-a4de-f0effa2261f1', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:14:51.175822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "5d5cfe70-1a06-4c64-91bf-4681779bd260", "tag": "bf337196-4336-4bb9-9fcf-4dfbcb7910a6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:51.187775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:51.188253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:51.188673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:51.202203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] Creating event network-changed:bf337196-4336-4bb9-9fcf-4dfbcb7910a6 for instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 on hv-cinder-85837 Sep 19 16:14:51.207924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-ef2857a5-9a45-4f4b-a115-e4fbe7ee846a req-86572140-3b6f-4146-b3a7-a8e52f356573 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037774 Sep 19 16:14:51.208548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 90/191] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:51.341395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:51.342212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:51.349884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:51.350662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:51.351299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:51.352386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:51.353120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:51.353730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:51.700791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:51.715260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:51.719192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:51.720868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ebd28637-470c-456d-9147-f99168589de8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1408 microversion: 2.20 time: 0.384102 Sep 19 16:14:51.735689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 91/192] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1408 bytes in 400 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:14:52.042692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:52.044179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:52.054462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:52.055352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:52.055884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:52.057343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:52.058197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:52.058908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:52.346162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:52.356349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-675bcc58-808f-461e-bcb4-62583c1b27af tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.317530 Sep 19 16:14:52.357276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 92/193] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:52 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:52.750336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:52.751120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:52.758747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:52.759985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:52.760569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:52.761500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:52.762306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:52.762931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:52.790677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:52.995492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:14:52.996057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:14:52.996366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:14:52.997422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:14:53.007420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:14:53.030544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.035750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.036738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.041448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.042220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.042853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.086782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:53.091257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Getting quotas for project 0e641d48680849328816261ffd5b3b42. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:14:53.104278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Getting quotas for user fe870f5b19334b47b972d7655876472e and project 0e641d48680849328816261ffd5b3b42. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:14:53.106034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:53.111270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:53.118172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-76f8751e-8cb0-4cfa-8c97-54b98a417639 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1600 microversion: 2.20 time: 0.372253 Sep 19 16:14:53.133267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 93/194] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:14:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1600 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:14:53.157085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:14:53.159598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:14:53.180817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:53.182137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:53.198976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.200416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.201905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.203564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.204993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.206492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.296768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4cd4619a-46f5-4c0b-9c9b-c328d114965d tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.292998 Sep 19 16:14:53.300840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 102/195] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:14:51 2022] POST /compute/v2.1/servers => generated 380 bytes in 2298 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:14:53.321015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-393e118f-dd01-48da-bdeb-f6283aeeb2aa tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:53.324169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-393e118f-dd01-48da-bdeb-f6283aeeb2aa tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:53.491290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-393e118f-dd01-48da-bdeb-f6283aeeb2aa tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:53.506661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-393e118f-dd01-48da-bdeb-f6283aeeb2aa tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1213 microversion: 2.1 time: 0.197224 Sep 19 16:14:53.509202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 103/196] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:53 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1213 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:53.510498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e6e91bea-a5f1-47e5-8cbc-1520261b8e48 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.337901 Sep 19 16:14:53.518809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 94/197] 10.222.0.57 () {60 vars in 1316 bytes} [Mon Sep 19 16:14:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-interface => generated 284 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:14:53.525560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:53.527000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:53.535632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.536515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.537699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.538658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:53.539316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:53.540746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:53.993566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:54.011342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-358ac9fc-044f-4de8-b6ad-34717eb223a3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.497795 Sep 19 16:14:54.013016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 104/198] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:53 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 502 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:54.565563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:54.567034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:54.578194 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:54.579738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:54.581222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:54.582749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:54.584517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:54.585500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:54.703007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:54.713435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fe842b6e-0ede-46e7-b4db-11b1080f06fe tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1265 microversion: 2.1 time: 0.155449 Sep 19 16:14:54.714901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 95/199] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:54 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1265 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:55.037429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:55.038252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:55.040913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:14:55.057392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:55.060120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:55.061556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:55.067634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:55.069274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:55.070179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:55.339885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "bae51cb6-fc0c-4343-9cd7-c04ef65d6b93", "tag": "84b6192c-6cd6-4692-a397-69b77dc536c9"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:55.369591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:55.370223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:55.371066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:55.435450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] Creating event network-changed:84b6192c-6cd6-4692-a397-69b77dc536c9 for instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 on hv-cinder-85837 Sep 19 16:14:55.452805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-bdf67ace-3e0b-4c0e-8696-d5f38e985291 req-d5c92c8b-038d-4761-ae19-c037282566e4 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.114814 Sep 19 16:14:55.456996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 96/200] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:55 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:55.704354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:55.724070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-284505e7-ea09-4c5d-8be8-14d82cd94fee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.691031 Sep 19 16:14:55.726060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 105/201] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:55 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:55.756964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:55.761485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:55.792833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:55.793893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:55.794938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:55.798714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:55.799740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:55.800597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:56.256056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:56.265676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9cfc55b1-8ff8-4b4e-9b0f-6ff2c4b259ad tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1283 microversion: 2.1 time: 0.527626 Sep 19 16:14:56.270296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 97/202] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:55 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1283 bytes in 536 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:56.775467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:56.776648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:56.793129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:56.794666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:56.800000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:56.802018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:56.806510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:56.807719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:57.311395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:57.313056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:57.329337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:57.329839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:57.330281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:57.331070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:57.331532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:57.332106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:57.519103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:57.544610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4c301b13-701a-49d8-b0d0-49a4f52b888a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.772248 Sep 19 16:14:57.548267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 106/203] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:56 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 777 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:57.558121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:57.577880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:57.578616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:57.579185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:57.596126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] Creating event network-changed:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:14:57.604932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-a31fb5de-c7c7-45f7-a54d-b57a5f11d6a9 req-84a40d14-813d-4495-9a3b-f9acc7cb523c service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.054960 Sep 19 16:14:57.605744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 107/204] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:57 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:14:57.902621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:57.928906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-06442fa4-57f9-4acb-b7aa-155fcc2b87af tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.621964 Sep 19 16:14:57.929669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 98/205] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:57 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:14:58.596913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:58.597860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:58.615946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:58.616467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:58.616915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:58.617852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:58.618285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:58.618773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:58.870990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "82016461-83f3-4a0f-8f3c-bc488a5f9656", "tag": "66e11cf2-55b0-4ec4-9468-0d2131b899f0"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:14:58.891131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:58.892031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:58.892624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:58.916338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] Creating event network-changed:66e11cf2-55b0-4ec4-9468-0d2131b899f0 for instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 on hv-cinder-85837 Sep 19 16:14:58.929504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-64dd535b-81d8-4607-a992-82f38922f2ea req-99f439bb-a929-456e-a8ae-f432c040a734 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066563 Sep 19 16:14:58.932448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 99/206] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:14:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:14:58.974982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:14:58.976775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:14:58.986860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:58.987787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:58.988470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:58.989417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:14:58.992533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:14:58.993595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:14:59.103271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:59.118452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-97a0f3b3-1d6a-4ec7-9c4e-38d8d285b4ad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.526817 Sep 19 16:14:59.121804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 108/207] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:58 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 531 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:14:59.259530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:14:59.269770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2e6ba989-afe3-4a13-a5f3-ad8ad9603089 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.302003 Sep 19 16:14:59.270826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 100/208] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:14:58 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:00.168463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:00.170730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:00.180204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:00.180825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:00.181533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:00.182922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:00.183680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:00.184376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:00.311131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:00.314637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:00.384346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:00.386888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:00.388809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:00.392830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:00.395841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:00.400452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:00.834603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:00.847676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-99a7108d-01b0-4f27-bf31-1ad487ef51e7 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.556226 Sep 19 16:15:00.850123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 101/209] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:00 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:01.002755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:01.012880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bd9cfb16-9628-4357-b681-62852a51462a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.849518 Sep 19 16:15:01.013684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 109/210] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:00 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 852 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:01.525676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:01.526631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:01.533475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.533973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.534454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.535180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.535635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.536055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.567011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:15:01.720781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ee043bb9-94bd-4fa5-a37b-d79988e91f4a tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "DELETE /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 204 len: 0 microversion: 2.1 time: 0.199230 Sep 19 16:15:01.722062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 102/211] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:15:01 2022] DELETE /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:15:01.732043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:01.733015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:01.740017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.740503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.740992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.741704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.742159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.742627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.903102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:01.903926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:01.911595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.912176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.912818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.913600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:01.914257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:01.914808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:01.936532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:01.945626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-27f7e88f-ac12-4657-aab5-40c33c1d14eb tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1553 microversion: 2.1 time: 0.217912 Sep 19 16:15:01.947099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 110/212] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:01 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1553 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:02.040403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:02.041131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:02.049885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:02.050430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:02.051069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:02.052799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:02.053639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:02.054227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:02.119972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:02.167549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c5c84d12-c888-432c-9151-8ea6a8cb4337 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.268714 Sep 19 16:15:02.169023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 103/213] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:01 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:02.234850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:02.245728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b81bfe20-3640-4bfd-8ad0-20660c7ac5e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.209710 Sep 19 16:15:02.246675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 111/214] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:02 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:02.982800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:02.984942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:02.995894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:02.996963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:02.997526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:02.998586 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:02.999387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:02.999963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:03.150207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:03.159466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3c9f2670-95dd-429c-b6fb-e051de2dce25 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1553 microversion: 2.1 time: 0.186636 Sep 19 16:15:03.160074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 104/215] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:02 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1553 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:03.195017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:03.195909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:03.200354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:03.207501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:03.209287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:03.209915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:03.210803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:03.211455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:03.212003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:03.275981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:03.277231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:03.289152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:03.290186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:03.290981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:03.292640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:03.293456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:03.294237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:03.475813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:03.485798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-40d133f4-f350-4cc3-9ee1-73e6c139c22f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.295784 Sep 19 16:15:03.487939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 112/216] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:03 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:03.519457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:03.528936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1e1d8baf-85f4-43df-b462-35c2f500d54c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.262012 Sep 19 16:15:03.529951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 105/217] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:03 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:04.189269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:04.191098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:04.195176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:04.211996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.213045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.214218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.216046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.216876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.217640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.226966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:04.469907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:04.497295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-069f5dd6-f1eb-4fcc-95b5-045d4c5d0815 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1553 microversion: 2.1 time: 0.314944 Sep 19 16:15:04.499248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 113/218] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:04 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1553 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:04.524207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:04.524833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:04.527522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:04.543461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.544290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.544937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.546134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.546832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.547856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.559038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:04.559875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:04.570770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.571814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.572740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.574549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:04.575525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:04.576408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:04.731964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:04.741425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5416bf1e-4aa0-40e7-a7e7-83a2c7e40eb2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.221375 Sep 19 16:15:04.742016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 106/219] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:04 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:04.757959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:04.773232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5c0f22de-ac9f-4925-81c5-568d6be5ea32 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.222752 Sep 19 16:15:04.774292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 114/220] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:04 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:05.540239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:05.543704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:05.556821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.558165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.559316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:05.561006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.562181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.563772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:05.766259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:05.767037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:05.775656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.776150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.776579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:05.777280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.777735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.783937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:05.913298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:05.927772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-71a8c0bf-400e-4603-92e5-9847fcdc4dd0 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1553 microversion: 2.1 time: 0.394458 Sep 19 16:15:05.928707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 107/221] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:05 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1553 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:05.954785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:05.956594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:05.972577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.975739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.976825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:05.978436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:05.979453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:05.980377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:06.257062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:06.273233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d207336b-7246-4c5e-9298-0e74803bb45b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.332033 Sep 19 16:15:06.274002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 108/222] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:05 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:06.411870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:06.440101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a437e7d6-3a1b-406a-a4e4-700090010612 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.677464 Sep 19 16:15:06.454950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 115/223] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:05 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 694 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:06.645390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82016461-83f3-4a0f-8f3c-bc488a5f9656", "name": "network-vif-plugged", "status": "completed", "tag": "66e11cf2-55b0-4ec4-9468-0d2131b899f0"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:06.658089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0d7d2812-f954-45c7-9916-4bd9fcb96bcc", "name": "network-vif-deleted", "tag": "88a993b9-866a-428d-a980-4b36a142d1f5"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:06.658994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:06.659852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:06.661117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:06.687553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:06.688475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:06.689278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:06.702826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] Creating event network-vif-plugged:66e11cf2-55b0-4ec4-9468-0d2131b899f0 for instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 on hv-cinder-85837 Sep 19 16:15:06.708711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-8c254c1a-90f1-4bad-8242-93be2483a356 req-abe9511c-d7cb-49dc-b4a1-b3a8d3016de6 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068748 Sep 19 16:15:06.709709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 109/224] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:06.716329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] Creating event network-vif-deleted:88a993b9-866a-428d-a980-4b36a142d1f5 for instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc on hv-cinder-85837 Sep 19 16:15:06.728659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-1cad23e9-e66c-4149-9862-9f15ba471625 req-ad23d3d9-064c-4e23-97af-22ff70ea191b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.079007 Sep 19 16:15:06.732834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 116/225] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:06.966853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:06.967673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:06.975740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:06.976339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:06.976852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:06.977654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:06.978200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:06.978762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:07.093728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:07.101757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:07.109502 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fbbcca7e-3d6b-41bd-b871-5896909bde0d tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1345 microversion: 2.1 time: 0.147341 Sep 19 16:15:07.110514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 110/226] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:06 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1345 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:07.314314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:07.315574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:07.327430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:07.328135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:07.328897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:07.330115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:07.331569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:07.332326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:07.500009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:07.502163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:07.519091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:07.520215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:07.521673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:07.523315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:07.524382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:07.525444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:07.658235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:07.676068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f5ecf136-0fc3-4f6e-9e0d-3175641532d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.369192 Sep 19 16:15:07.677151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 117/227] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:07 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 372 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:07.878924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:07.891634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9f69be88-39aa-4f6e-9dc7-08440621f600 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.398904 Sep 19 16:15:07.892397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 111/228] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:07 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:08.153895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:08.155971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:08.197201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.203152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.205988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.208992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.213250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.215393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.329535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.330662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.330662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.331127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lazy-loading 'fault' on Instance uuid 0d7d2812-f954-45c7-9916-4bd9fcb96bcc {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:15:08.452093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:08.465029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f8f4ea64-48fc-4e4d-810e-179dca1742bf tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1350 microversion: 2.1 time: 0.321953 Sep 19 16:15:08.465692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 118/229] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:08 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1350 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:08.729645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:08.730769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:08.742340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.743596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.744361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.745603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.746747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.747790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.916563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:08.917610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:08.932209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.934137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.935735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:08.936853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:08.939652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:08.940474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:09.025924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:09.043460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-462be6d4-5a96-400a-9809-c02d84cb9136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.333094 Sep 19 16:15:09.044912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 112/230] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:08 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:09.273426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:09.284844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-045b3130-57ce-457e-afc6-ad0f134666f4 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1313 microversion: 2.1 time: 0.372562 Sep 19 16:15:09.292825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 119/231] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:08 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1313 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:09.493900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:09.495542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:09.516297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:09.517906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:09.519696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:09.523823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:09.525127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:09.526243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:09.601141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:09.601605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:09.602346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:09.603095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lazy-loading 'fault' on Instance uuid 0d7d2812-f954-45c7-9916-4bd9fcb96bcc {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:15:09.971879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '428a95af27d2468bb39391ca3eefea40', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '34baf55057ee42ffbd099de2d65544a1', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:09.990130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e880db5d-a2f6-42ad-826b-554f54e5a7d9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 200 len: 1304 microversion: 2.1 time: 0.503868 Sep 19 16:15:09.991152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 113/232] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:09 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 1304 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:10.077492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:10.078720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:10.090982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:10.090982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:10.090982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:10.098670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:10.100190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:10.100942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:10.322352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:10.325703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:10.343862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:10.344560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:10.346036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:10.347063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:10.348201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:10.348971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:10.383172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:10.416546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2c7332b5-3c2f-494a-bfa0-89b57458c450 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.345213 Sep 19 16:15:10.417640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 120/233] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:10 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:10.588604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:10.600599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7444440c-0e97-429b-9418-30d1467ca580 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1496 microversion: 2.1 time: 0.287181 Sep 19 16:15:10.601599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 114/234] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:10 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1496 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:11.028977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:11.030305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:11.044599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.045652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.046492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.047985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.049088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.050055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.084755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] HTTP exception thrown: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Request method failure captured: Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc HTTP/1.1 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU2hHeuw8nRAVh3Y4-9oiNezI7aKNA6tDr9-rcmvq35EcrzVS92Ad8yQxVelRMmVY4rjL-wfO3YX4amCUHiLO7lGBhtKtjCUFRReBBwrPaLA8_k-KCMiy2NPl_wJOp-g96XCvUGneSmiO2E22k6x3gcktJPmCf1TXQlM7hV1qCB_2o Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:15:11.098290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_34baf55057ee42ffbd099de2d65544a1"}]}] Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 428a95af27d2468bb39391ca3eefea40 Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.100939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '0d7d2812-f954-45c7-9916-4bd9fcb96bcc'} Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:11.101779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.104200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.106224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Returning 404 to user: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:11.110219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-01666f15-428d-404e-b9bf-bd6c76049f56 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 404 len: 111 microversion: 2.1 time: 0.086237 Sep 19 16:15:11.114188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 121/235] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:11 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 111 bytes in 94 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:15:11.251293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:11.252018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:11.264439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.265395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.266136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.267378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.268228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.268951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.324472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:15:11.343554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:11.348594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:11.363490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.365783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.367068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.369066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.370496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.371626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.400978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] HTTP exception thrown: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Request method failure captured: Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc/os-volume_attachments HTTP/1.1 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU2hHeuw8nRAVh3Y4-9oiNezI7aKNA6tDr9-rcmvq35EcrzVS92Ad8yQxVelRMmVY4rjL-wfO3YX4amCUHiLO7lGBhtKtjCUFRReBBwrPaLA8_k-KCMiy2NPl_wJOp-g96XCvUGneSmiO2E22k6x3gcktJPmCf1TXQlM7hV1qCB_2o Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:15:11.406078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_34baf55057ee42ffbd099de2d65544a1"}]}] Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 428a95af27d2468bb39391ca3eefea40 Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.408492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '0d7d2812-f954-45c7-9916-4bd9fcb96bcc'} Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:11.409629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:11.418268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:11.423297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Returning 404 to user: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:11.423297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a6cb3bea-ba57-41a4-963b-f6fdd37cd20b tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.075907 Sep 19 16:15:11.423297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 122/236] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:15:11 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc/os-volume_attachments => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:15:11.463285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:11.464939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:11.492309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.495673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.497229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.498561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.499351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.500332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.510840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a3976058-efbd-41b9-9083-5dec001318fd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "DELETE /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 204 len: 0 microversion: 2.1 time: 0.263717 Sep 19 16:15:11.512037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 115/237] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:15:11 2022] DELETE /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:15:11.538136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:11.538136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:11.559459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.560881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.561685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.563157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:11.563830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:11.564578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:11.808425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:11.820487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-75688de6-f557-40f0-9066-e3f166c78bd6 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1502 microversion: 2.1 time: 0.298678 Sep 19 16:15:11.835490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 116/238] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:11 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1502 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:11.880956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:11.897428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a875072d-f6ea-4e50-b41f-cf1349cf9080 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.437896 Sep 19 16:15:11.898938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 123/239] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:11 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 442 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:12.850990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:12.852297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:12.864937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:12.866142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:12.866983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:12.868474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:12.869355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:12.870354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:12.909082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:12.910430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:12.922467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:12.923414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:12.924256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:12.926403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:12.928384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:12.929249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:12.956915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] HTTP exception thrown: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Request method failure captured: Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc HTTP/1.1 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU2hHeuw8nRAVh3Y4-9oiNezI7aKNA6tDr9-rcmvq35EcrzVS92Ad8yQxVelRMmVY4rjL-wfO3YX4amCUHiLO7lGBhtKtjCUFRReBBwrPaLA8_k-KCMiy2NPl_wJOp-g96XCvUGneSmiO2E22k6x3gcktJPmCf1TXQlM7hV1qCB_2o Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:15:12.960453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_34baf55057ee42ffbd099de2d65544a1"}]}] Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 428a95af27d2468bb39391ca3eefea40 Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:12.963410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '0d7d2812-f954-45c7-9916-4bd9fcb96bcc'} Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:12.964915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:12.968169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:12.973575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Returning 404 to user: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:12.978299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2226a781-fadb-4b8b-8b02-2c253efa905f tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "DELETE /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 404 len: 111 microversion: 2.1 time: 0.072483 Sep 19 16:15:12.984894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 124/240] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:15:12 2022] DELETE /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 111 bytes in 81 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:15:12.997030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:12.998633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:13.015932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:13.017176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:13.018168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:13.019820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:13.022444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:13.023571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:13.150035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:13.164373 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f2452aa0-294d-4454-974b-d88e8c7d3a07 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1502 microversion: 2.1 time: 0.320977 Sep 19 16:15:13.165267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 117/241] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:12 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1502 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:13.174808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:13.176394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] [instance: 0d7d2812-f954-45c7-9916-4bd9fcb96bcc] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:13.188348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:13.189782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:13.190942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:13.193151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:13.194435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:13.195276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:13.256825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] HTTP exception thrown: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Request method failure captured: Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc HTTP/1.1 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU2hHeuw8nRAVh3Y4-9oiNezI7aKNA6tDr9-rcmvq35EcrzVS92Ad8yQxVelRMmVY4rjL-wfO3YX4amCUHiLO7lGBhtKtjCUFRReBBwrPaLA8_k-KCMiy2NPl_wJOp-g96XCvUGneSmiO2E22k6x3gcktJPmCf1TXQlM7hV1qCB_2o Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:15:13.266835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/34baf55057ee42ffbd099de2d65544a1"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_34baf55057ee42ffbd099de2d65544a1"}]}] Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 34baf55057ee42ffbd099de2d65544a1 Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-DeleteServersTestJSON-608573563 Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 428a95af27d2468bb39391ca3eefea40 Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-DeleteServersTestJSON-608573563-project Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:13.269191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '0d7d2812-f954-45c7-9916-4bd9fcb96bcc'} Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:13.270582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. Sep 19 16:15:13.273184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:13.278427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] Returning 404 to user: Instance 0d7d2812-f954-45c7-9916-4bd9fcb96bcc could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:13.278427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4d2dd5bd-2161-46ea-9c74-e7183c0fcbd9 tempest-DeleteServersTestJSON-608573563 tempest-DeleteServersTestJSON-608573563-project] 10.222.0.57 "GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc" status: 404 len: 111 microversion: 2.1 time: 0.102475 Sep 19 16:15:13.278427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 118/242] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:13 2022] GET /compute/v2.1/servers/0d7d2812-f954-45c7-9916-4bd9fcb96bcc => generated 111 bytes in 105 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:15:13.412773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:13.425525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aa557a26-db19-45e4-bf23-844fada651f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.438404 Sep 19 16:15:13.426426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 125/243] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:12 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:14.220140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:14.222832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:14.237836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:14.240721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:14.242410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:14.244320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:14.245494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:14.246671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:14.462340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:14.464912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:14.479412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:14.480764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:14.481904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:14.484737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:14.485985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:14.487126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:14.935491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:14.996419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:15.012979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7d88cb41-7891-4675-8f92-745db604431b tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1502 microversion: 2.1 time: 0.797514 Sep 19 16:15:15.018840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 119/244] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:14 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1502 bytes in 811 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:15.023277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ae6688d-ddaa-41df-9e44-03ea99f76b2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.570980 Sep 19 16:15:15.024322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 126/245] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:14 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 573 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:15.366840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "82016461-83f3-4a0f-8f3c-bc488a5f9656", "name": "network-vif-deleted", "tag": "66e11cf2-55b0-4ec4-9468-0d2131b899f0"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:15.384187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:15.391993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:15.392748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:15.442552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] Creating event network-vif-deleted:66e11cf2-55b0-4ec4-9468-0d2131b899f0 for instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 on hv-cinder-85837 Sep 19 16:15:15.452291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-4258ad8f-b76b-49e4-8008-83efb422835b req-dd246da1-adbf-4fea-b4f9-ce2033c58ef1 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.089538 Sep 19 16:15:15.453141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 120/246] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:16.051319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:16.061117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:16.065653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:16.066419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:16.080305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:16.080812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:16.081254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:16.081976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:16.084620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:16.085343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:16.102855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:16.105320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:16.106633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:16.109858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:16.122107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:16.123245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:16.208306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:16.209306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:16.209913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:16.210559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lazy-loading 'fault' on Instance uuid 82016461-83f3-4a0f-8f3c-bc488a5f9656 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:15:16.614881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:16.643585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-77403dbd-038e-4f36-99d2-ab02af325d2f tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 200 len: 1303 microversion: 2.1 time: 0.605587 Sep 19 16:15:16.659184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:16.666806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 127/247] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:16 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 1303 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:16.702179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-860904ad-a64a-4d4a-9536-e3a08cab55a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.640723 Sep 19 16:15:16.705836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 121/248] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:16 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 647 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:17.703247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:17.703907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 82016461-83f3-4a0f-8f3c-bc488a5f9656] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:17.728635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:17.740982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:17.742070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:17.746070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:17.747089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:17.747900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:17.750543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:17.751157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:17.768029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:17.768787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:17.769383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:17.779953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:17.781308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:17.781917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:17.798884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] HTTP exception thrown: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Request method failure captured: Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 HTTP/1.1 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJV3txAxM0e1pa_omUrKlxntuQcUwHfLnY2Xi5zQoDtOhYwavJqWdzrt68rLnESYdC_zoTviGjZSkHn1zATznv0qaXJHTBV8eGW3WVK6X342HA90gnnaXPUcaGowbpG_frPpl7Hos1KshodOh924_W7p07c8hatJJ6BaNmdSH34BNYk Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e641d48680849328816261ffd5b3b42 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesActionsTest-303283582 Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:15:17.822173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e641d48680849328816261ffd5b3b42"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e641d48680849328816261ffd5b3b42"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e641d48680849328816261ffd5b3b42"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e641d48680849328816261ffd5b3b42"}]}] Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesActionsTest-303283582 Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e641d48680849328816261ffd5b3b42 Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesActionsTest-303283582 Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesActionsTest-303283582-project Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fe870f5b19334b47b972d7655876472e Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesActionsTest-303283582-project Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. Sep 19 16:15:17.824910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '82016461-83f3-4a0f-8f3c-bc488a5f9656'} Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:17.826447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. Sep 19 16:15:17.828799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:15:17.832491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Returning 404 to user: Instance 82016461-83f3-4a0f-8f3c-bc488a5f9656 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:17.834517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b720a845-edc0-4f5c-b0f2-5c09cf66ca2e tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656" status: 404 len: 111 microversion: 2.1 time: 0.135912 Sep 19 16:15:17.850742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 128/249] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:17 2022] GET /compute/v2.1/servers/82016461-83f3-4a0f-8f3c-bc488a5f9656 => generated 111 bytes in 153 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:15:18.138288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:18.143456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1216522655", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "networks": [{"uuid": "67cc7387-dada-400e-a4de-f0effa2261f1"}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:18.151842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d625f661-6498-4ae3-bf0f-120cf9bb9bcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.403203 Sep 19 16:15:18.153618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 122/250] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:17 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:18.381165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] validate_networks() for [('67cc7387-dada-400e-a4de-f0effa2261f1', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:15:19.194051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:19.194756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:19.217019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:19.218357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:19.219318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:19.222468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:19.226065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:19.229227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:19.306831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:15:19.308210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:15:19.309221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:15:19.310085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:15:19.310805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:15:19.330954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:19.336775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:19.337697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:19.338891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:19.339674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:19.340485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:19.407999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Getting quotas for project 0e641d48680849328816261ffd5b3b42. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:15:19.426034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Getting quotas for user fe870f5b19334b47b972d7655876472e and project 0e641d48680849328816261ffd5b3b42. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:15:19.463682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:15:19.470408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:15:19.622954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-56dd6110-da8b-40f4-8d7a-4c9c69f7ec39 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.486197 Sep 19 16:15:19.627725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 129/251] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:15:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 1492 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:15:19.651024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1b1b3d8c-3e93-4c20-be91-8f5738206817 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:19.653729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1b1b3d8c-3e93-4c20-be91-8f5738206817 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:19.979860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1b1b3d8c-3e93-4c20-be91-8f5738206817 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:19.992287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1b1b3d8c-3e93-4c20-be91-8f5738206817 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1214 microversion: 2.1 time: 0.343824 Sep 19 16:15:19.993527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 130/252] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:19 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1214 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:20.273960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:20.304929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-56fa4043-4edc-46e9-98fd-dadfa38d70a2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 1.121338 Sep 19 16:15:20.315551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 123/253] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:19 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 1133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:21.031110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:21.033157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:21.053830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:21.054498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:21.055677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:21.056560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:21.057026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:21.057452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:21.313900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:21.335329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:21.336606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:21.346790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-868ae63f-8a28-43bf-b03c-55115985b2dd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1210 microversion: 2.1 time: 0.311115 Sep 19 16:15:21.349599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 131/254] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:21 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1210 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:21.352201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:21.354191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:21.356157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:21.357878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:21.359052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:21.360461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:21.936822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:21.957047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ae4d1894-d415-4571-9270-88a33827b1fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.625490 Sep 19 16:15:21.960294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 124/255] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:21 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:22.389672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:22.396543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:22.418310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:22.424198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:22.425415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:22.428909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:22.430688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:22.433218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:22.603200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:22.622302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d501b64d-a2e3-4c1a-ad76-3fa8de9dacac tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1274 microversion: 2.1 time: 0.240037 Sep 19 16:15:22.624714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 132/256] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:22 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1274 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:23.003245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:23.005995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:23.021861 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:23.022519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:23.023202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:23.024098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:23.027668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:23.028139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:23.385991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:23.400827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e1d6240-f989-44cd-8881-9d3055e4e037 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.403076 Sep 19 16:15:23.402338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 125/257] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:22 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 406 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:23.659178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:23.659802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:23.677222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:23.677973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:23.678468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:23.679868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:23.680360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:23.680853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:23.949431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:23.961040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-56a96943-f885-428a-a61c-05103f92cbe9 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.305835 Sep 19 16:15:23.963818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 133/258] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:23 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:24.426215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:24.427561 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:24.435722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:24.449909 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:24.450927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:24.453881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:24.458928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:24.461983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:24.850698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:24.875760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:24.910142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6bcf7830-75d9-4fb4-862b-a3d3ebd2163e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.481696 Sep 19 16:15:24.912424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 126/259] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:24 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 491 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:24.996984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:24.998194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:25.007886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:25.013694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:25.015048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:25.016336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:25.019084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:25.020970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:25.090897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:25.899433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:25.915716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-13d8005c-2bf6-45a9-98a9-09cfb39fac09 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.926043 Sep 19 16:15:25.918718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 134/260] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:24 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 931 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:25.940461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:25.941320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:25.959781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:25.960856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:25.961669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:25.962726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:25.963466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:25.964124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:26.554284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4e3c3b52-011e-4545-bd78-82eb5a63673c", "tag": "b19d131c-2b9e-488d-9786-01bc72f673d4"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:26.614618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=77,nova_api:INSERT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:26.629106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:26.630598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:26.631262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:26.738312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] Creating event network-changed:b19d131c-2b9e-488d-9786-01bc72f673d4 for instance 4e3c3b52-011e-4545-bd78-82eb5a63673c on hv-cinder-85837 Sep 19 16:15:26.749528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-14f3e9ca-d2f5-4b8c-ac80-4f67e9216968 req-dcbd5bf6-98c8-4996-bdfa-6ee2dab59082 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.205300 Sep 19 16:15:26.752082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 135/261] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:26.948624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:26.950234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:26.974458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:26.977554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:26.978018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:26.993963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:26.996812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:27.006632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:27.181914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:27.233335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-42097780-62dd-4e86-9a31-0fb73cd5cf49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 1.296839 Sep 19 16:15:27.240354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 127/262] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:25 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 1305 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:27.526087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:27.545364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-896c9314-961d-4555-9ad1-710716512054 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.603400 Sep 19 16:15:27.558442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 136/263] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:26 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:28.266331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:28.267545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:28.288519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:28.289725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:28.290550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:28.292059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:28.293115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:28.294153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:28.518914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:28.532399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-95877585-abc8-4711-a66a-6c76f80fd74c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.273129 Sep 19 16:15:28.533848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 128/264] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:28 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:28.597284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:28.598922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:28.605574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:28.606657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:28.607245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:28.608349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:28.609429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:28.610038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:28.928881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:28.967645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f791e53-975a-487d-af73-2ad904b40d31 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.374113 Sep 19 16:15:28.978496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 137/265] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:28 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 379 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:29.566665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:29.570643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:29.618087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:29.618990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:29.619741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:29.620951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:29.621729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:29.622567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:30.015841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:30.016924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:30.033073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:30.035138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:30.042040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:30.044887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:30.046175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:30.047003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:30.047592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:30.048034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:30.123411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:30.138971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-332ddb6f-6728-4207-9244-333568006fe9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1520 microversion: 2.1 time: 0.581340 Sep 19 16:15:30.178913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 129/266] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:29 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1520 bytes in 624 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:30.609676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:30.627696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58ab7452-47c5-4823-a481-78cd48a8f471 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.617014 Sep 19 16:15:30.668432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 138/267] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:30 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 658 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:31.187619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:31.188690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:31.204035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.210146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.210816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.211798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.212530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.224060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.583726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:31.600196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1b4eddc4-477d-42a2-8809-e7f2d95415bc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1500 microversion: 2.1 time: 0.420949 Sep 19 16:15:31.607911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 130/268] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:31 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1500 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:31.624041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:31.624874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:31.631940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.632442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.632884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.633612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.634067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.634733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.675824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:31.676966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:31.683654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.684144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.684755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.685673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:31.686310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:31.686937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:31.904185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:31.917377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6a823876-3337-4bcd-a4e1-b61d4df192d4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1516 microversion: 2.1 time: 0.297304 Sep 19 16:15:31.918430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 139/269] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:31 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1516 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:31.983108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:31.993850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1bd9df24-7148-49e5-bfed-99b35d69d5b5 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.321848 Sep 19 16:15:31.995279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 131/270] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:31 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:33.032569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:33.033983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:33.044777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:33.046086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:33.046811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:33.047541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:33.048163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:33.048735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:33.221037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c9810317-dd4f-4ab9-948e-7b56fd4872c2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Action: 'create', calling method: .version_select at 0x7fadbfab9e50>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1392065781", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCpp/Qzgk/mRZmziKuZD8oKTvO7UhCmycDxXg9oUgYuaagcmQt6Zt77PTe+AYILHW0KJbfqRkpi2FJop1/5hMnG2/5WRrN3b9Ts07L0AjxAZN0tqHaYlqEIhvho0hYrVOg=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:33.253369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-c9810317-dd4f-4ab9-948e-7b56fd4872c2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Getting quotas for project 2a726afa4ec84ec99a5b1bd08c95a0cd. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:15:33.266602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-c9810317-dd4f-4ab9-948e-7b56fd4872c2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Getting quotas for user 9fd6c22a61be43658ae87052b867e384 and project 2a726afa4ec84ec99a5b1bd08c95a0cd. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:15:33.303997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c9810317-dd4f-4ab9-948e-7b56fd4872c2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.419710 Sep 19 16:15:33.305186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 140/271] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:15:32 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:33.309583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:33.320148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-caac5c46-83d3-4e35-97ad-707a85fa17cc tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.291561 Sep 19 16:15:33.320789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 132/272] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:33 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 293 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:34.344454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:34.345194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:34.351984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:34.352825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:34.353434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:34.354396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:34.355026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:34.355615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:34.695368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:34.710671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-158ad3b2-705a-48ce-9309-b6c89a53ed24 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.367124 Sep 19 16:15:34.728273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 141/273] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:34 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:35.741569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:35.744904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:35.795488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=61,nova_api:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:35.802031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:35.805512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:35.806657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:35.807908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:35.814046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:35.815170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:36.115257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:36.117043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:36.125531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:36.126177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:36.126776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:36.135907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:36.141150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:36.141762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:36.188173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'flavor' on Instance uuid c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:15:36.239506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:36.273988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5f956092-da9b-4c64-8b8c-9e50073714a2 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.540370 Sep 19 16:15:36.287204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 133/274] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:35 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 555 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:36.389850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b27a3c8a-bae4-480a-ba3e-1ade1926bee2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/action" status: 200 len: 32 microversion: 2.1 time: 0.279851 Sep 19 16:15:36.394581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 142/275] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:15:36 2022] POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/action => generated 32 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 19 16:15:36.403947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:36.404897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:36.415526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:36.417657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:36.420566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:36.421567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:36.422891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:36.427478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:37.005097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:37.017098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0384b28b-a5d4-4988-8045-96d078cc2ac9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.617839 Sep 19 16:15:37.018864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 134/276] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:36 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 620 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:37.365079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e3c3b52-011e-4545-bd78-82eb5a63673c", "name": "network-vif-plugged", "status": "completed", "tag": "b19d131c-2b9e-488d-9786-01bc72f673d4"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:37.368816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:37.370114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:37.392766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:37.393284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:37.393758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:37.403837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:37.406337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:37.407101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:37.408354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:37.414667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:37.415380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:37.433393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] Creating event network-vif-plugged:b19d131c-2b9e-488d-9786-01bc72f673d4 for instance 4e3c3b52-011e-4545-bd78-82eb5a63673c on hv-cinder-85837 Sep 19 16:15:37.441774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-20ec3423-5e5e-4076-b96d-a27d7af6ce11 req-d5e4d910-9d80-4798-8a6d-87bb7e152913 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.084371 Sep 19 16:15:37.448596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 135/277] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:15:37.810559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:37.862277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-448bc247-988a-40f1-8f57-c455812f31ca tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.500797 Sep 19 16:15:37.881001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 143/278] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:37 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:38.057147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:38.058924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:38.075179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:38.076709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:38.078116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:38.078941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:38.079420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:38.080197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:38.519325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:38.529243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ffd0c082-b764-4f55-94ff-f6761b3e7198 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.485558 Sep 19 16:15:38.530291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 136/279] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:38 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 488 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:38.893520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:38.894365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:38.916455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:38.916953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:38.917599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:38.918694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:38.919919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:38.921214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:39.107001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:39.118818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58c9d2e9-2260-4da8-a4db-fae061508ca8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1314 microversion: 2.1 time: 0.233607 Sep 19 16:15:39.120467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 144/280] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:38 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1314 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:39.555571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:39.556611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:39.564621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:39.565299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:39.565886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:39.567159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:39.567782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:39.568424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:39.782623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:39.797155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e785c89-b05b-477b-806c-5d58b8cddc87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.246591 Sep 19 16:15:39.802885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 137/281] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:39 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:40.158190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:40.167619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:40.194107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:40.198794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:40.199408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:40.200632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:40.201437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:40.202164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:40.506068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:40.523176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-90af3c6a-a5f0-4e29-9191-5f59573d55ff tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1498 microversion: 2.1 time: 0.370669 Sep 19 16:15:40.524618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 145/282] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:40 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1498 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:40.842429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:40.843910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:40.856022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:40.857212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:40.858244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:40.859991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:40.860928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:40.861981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:41.121415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:41.136724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d429b44e-74c6-4dd0-bf6d-d192b746c672 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.303914 Sep 19 16:15:41.147209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 138/283] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:40 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 316 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:41.572604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:41.573456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:41.579691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:41.580177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:41.580618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:41.581349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:41.581798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:41.582360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:41.616013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:15:41.743474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-92d93e89-5d42-485e-a587-8669a9b9caa0 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 204 len: 0 microversion: 2.1 time: 0.174514 Sep 19 16:15:41.744085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 146/284] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:15:41 2022] DELETE /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:15:41.752590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:41.753307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:41.762161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:41.762709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:41.763153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:41.763879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:41.764339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:41.764768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:41.944359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:41.952978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3e473beb-8e12-4bf4-b612-d7fa560cd313 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1504 microversion: 2.1 time: 0.204267 Sep 19 16:15:41.953577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 139/285] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:41 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1504 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:42.190283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:42.192228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:42.203128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:42.203976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:42.204753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:42.206003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:42.207697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:42.207697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:42.399309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:42.411621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-51c9c76a-6b11-4b45-bda7-55b7d5235211 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.232931 Sep 19 16:15:42.411621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 147/286] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:42 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:42.982313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:42.983445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:42.996150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:42.996955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:42.997623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:42.998404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:42.998886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:42.999339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:43.176584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:43.184827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2e50f413-afb0-4281-acdb-a21e12196f28 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1504 microversion: 2.1 time: 0.209861 Sep 19 16:15:43.185479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 140/287] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:42 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1504 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:43.432698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:43.433365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:43.435438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:43.441488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:43.441981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:43.442510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:43.443238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:43.443699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:43.444128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:43.622790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:43.626634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:43.633766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c371633f-1154-4d74-b111-82e251b7da6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.204897 Sep 19 16:15:43.646757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 148/288] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:43 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:44.219788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:44.222687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:44.244549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:44.247532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:44.248795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:44.250490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:44.251684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:44.252475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:44.554170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:44.570465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ae203893-860f-4a15-827f-8b44a591c9f8 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1504 microversion: 2.1 time: 0.364358 Sep 19 16:15:44.571546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 141/289] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:44 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1504 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:44.666001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:44.667291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:44.681513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:44.682713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:44.683683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:44.684965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:44.685801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:44.686645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:44.720138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4e3c3b52-011e-4545-bd78-82eb5a63673c", "name": "network-vif-deleted", "tag": "b19d131c-2b9e-488d-9786-01bc72f673d4"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:44.735206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:44.736077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:44.736727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:44.762277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] Creating event network-vif-deleted:b19d131c-2b9e-488d-9786-01bc72f673d4 for instance 4e3c3b52-011e-4545-bd78-82eb5a63673c on hv-cinder-85837 Sep 19 16:15:44.771019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-42db2238-bcc5-40ef-aac5-3b5ed6a524d1 req-648fccb0-ed73-43ed-bd43-d614bf40a84d service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.058284 Sep 19 16:15:44.772538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 142/290] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:15:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:15:44.957200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:44.972585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-de1692dd-f993-4b67-8bfb-b4ab910b4136 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.317223 Sep 19 16:15:44.974176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 149/291] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:44 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:45.612233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:45.614402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:45.621687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:45.622537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:45.622812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:45.624060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:45.637069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:45.637558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:45.697872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:45.698729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:45.699479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:45.700220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lazy-loading 'fault' on Instance uuid 4e3c3b52-011e-4545-bd78-82eb5a63673c {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:15:45.776870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fe870f5b19334b47b972d7655876472e', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e641d48680849328816261ffd5b3b42', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:45.789874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e22d0d93-d308-428f-8a5a-e2ff594cf529 tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 200 len: 1304 microversion: 2.1 time: 0.182471 Sep 19 16:15:45.790832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 143/292] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:45 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 1304 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:46.013057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:46.014777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:46.026693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:46.027504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:46.029227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:46.033609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:46.033609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:46.033609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:46.242478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:46.258335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1e07c152-ca87-40d4-bac0-ea859fb62d0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.251483 Sep 19 16:15:46.259455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 150/293] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:46 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:46.833467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:46.834933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] [instance: 4e3c3b52-011e-4545-bd78-82eb5a63673c] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:46.855115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:46.856523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:46.857620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:46.859012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:46.859783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:46.860625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:46.879308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] HTTP exception thrown: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Request method failure captured: Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c HTTP/1.1 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJV3txAxM0e1pa_omUrKlxntuQcUwHfLnY2Xi5zQoDtOhYwavJqWdzrt68rLnESYdC_zoTviGjZSkHn1zATznv0qaXJHTBV8eGW3WVK6X342HA90gnnaXPUcaGowbpG_frPpl7Hos1KshodOh924_W7p07c8hatJJ6BaNmdSH34BNYk Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e641d48680849328816261ffd5b3b42 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesActionsTest-303283582 Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:15:46.880994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e641d48680849328816261ffd5b3b42"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e641d48680849328816261ffd5b3b42"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e641d48680849328816261ffd5b3b42"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e641d48680849328816261ffd5b3b42"}]}] Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesActionsTest-303283582 Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e641d48680849328816261ffd5b3b42 Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesActionsTest-303283582 Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesActionsTest-303283582-project Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fe870f5b19334b47b972d7655876472e Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesActionsTest-303283582-project Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. Sep 19 16:15:46.882563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '4e3c3b52-011e-4545-bd78-82eb5a63673c'} Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:15:46.883641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. Sep 19 16:15:46.885138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:15:46.886362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] Returning 404 to user: Instance 4e3c3b52-011e-4545-bd78-82eb5a63673c could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:15:46.886362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f21d7d2b-4b7b-49b1-87a3-b96883bd0acd tempest-VolumesActionsTest-303283582 tempest-VolumesActionsTest-303283582-project] 10.222.0.57 "GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c" status: 404 len: 111 microversion: 2.1 time: 0.060189 Sep 19 16:15:46.886362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 144/294] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:46 2022] GET /compute/v2.1/servers/4e3c3b52-011e-4545-bd78-82eb5a63673c => generated 111 bytes in 62 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:15:47.294603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:47.296176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:47.308222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:47.309640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:47.310935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:47.311956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:47.312506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:47.312879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:47.510161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:47.518407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c62e81a-e1a0-46e5-b816-5517eaccfb9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.231152 Sep 19 16:15:47.518979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 151/295] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:47 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:47.585966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:47.587345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:47.597123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:47.598552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:47.599164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:47.600135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:47.600712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:47.601252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:47.606661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:47.639562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0900be41-9910-4b05-bab1-a4879b368ee6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/ips" status: 200 len: 149 microversion: 2.1 time: 0.059999 Sep 19 16:15:47.642460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 145/296] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:15:47 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/ips => generated 149 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:15:48.567599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:48.568664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:48.579806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:48.580766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:48.583080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:48.583687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:48.584272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:48.585030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:48.995107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:49.008082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ea0e604-1377-4f03-b160-bb435be47baa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.455768 Sep 19 16:15:49.009103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 152/297] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:48 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:49.358355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "652d6b02-71d3-4c9f-a512-1e3c3c0ce750"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:15:49.360763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:49.367727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:49.368289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:49.368729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:49.369439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:49.369903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:49.370332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:50.033516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:50.034240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:50.041154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:50.041668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:50.042212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:50.043003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:50.043479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:50.043964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:50.218572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:50.227864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9b915922-17d3-47c3-a97c-09a79f7ab27b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.198369 Sep 19 16:15:50.228441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 153/298] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:50 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:51.132270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:51.132973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:51.139143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:51.139639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:51.140074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:51.140783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:51.141232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:51.141657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:51.168271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c90c51d0-0e77-45e4-9d98-c9339180605e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/ips" status: 200 len: 146 microversion: 2.1 time: 0.040612 Sep 19 16:15:51.168844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 154/299] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:15:51 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/ips => generated 146 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:15:51.253545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:51.254154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:51.261112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:51.261592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:51.262022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:51.262757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:51.263202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:51.263720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:51.429753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:51.438319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9bfcdce0-02e4-4370-94c3-e23c0f8c04e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.188894 Sep 19 16:15:51.438967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 155/300] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:51 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:52.388946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-db3c4022-dbb1-4990-b138-8702938b9687 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 3.034360 Sep 19 16:15:52.389652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 146/301] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:15:49 2022] POST /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 194 bytes in 3036 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:15:52.466114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:52.467179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:52.482978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:52.483814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:52.484544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:52.485834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:52.486597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:52.487485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:52.710252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:52.719799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc9086f9-a02d-4f3c-b55c-fc596b01de0e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.261196 Sep 19 16:15:52.720973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 156/302] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:52 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:53.752920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:53.753810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:53.760451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:53.760945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:53.761384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:53.762107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:53.762631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:53.763116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:53.917366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:53.931853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1544a102-a6b2-4892-898f-89132bce05fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.183688 Sep 19 16:15:53.932741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 147/303] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:53 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:54.967698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:54.968438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:54.976698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:54.977539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:54.978269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:54.979533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:54.980319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:54.981057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:55.151095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:55.162873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-60565471-80bc-4826-a208-b23622e589ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.198883 Sep 19 16:15:55.163468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 157/304] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:54 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:56.187109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:56.188265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:56.196034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:56.196533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:56.196970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:56.197674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:56.198125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:56.198613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:56.380192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:56.399802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a0d1dd85-77fd-4100-a71b-623cfbab5a46 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.216826 Sep 19 16:15:56.400963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 148/305] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:56 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:15:57.440438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:57.441163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:57.443217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=60,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:15:57.449374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:57.449964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:57.450529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:57.451326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:57.451854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:57.452498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:57.620925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:57.630433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-db1e7337-3ccf-4080-9012-9d2dd74c3fff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.194003 Sep 19 16:15:57.631101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 158/306] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:57 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:58.655367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:58.656431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:58.664762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:58.665448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:58.665944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:58.666750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:58.667247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:58.667736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:58.844574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:15:58.853585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e4083ffe-d876-4954-bd5e-291c1bc7a6ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.202417 Sep 19 16:15:58.854222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 149/307] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:58 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:15:59.882921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:15:59.883575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:15:59.895501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:59.896397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:59.897188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:15:59.898620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:15:59.899378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:15:59.900177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:00.136902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:00.151048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-419e4d57-e052-4525-9fc5-1944a828e7cc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.275205 Sep 19 16:16:00.151048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 159/308] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:15:59 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:00.236312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a330688a-4c62-4472-a9e9-6ec347d71d1e"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:00.239919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:00.247444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:00.247938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:00.248373 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:00.249113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:00.249568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:00.249994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:00.632693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cfd65809-92fc-49b8-a657-e49ccb4e1fe5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.401384 Sep 19 16:16:00.633312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 150/309] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:16:00 2022] POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments => generated 194 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:01.195422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:01.196621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:01.211244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:01.212114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:01.212862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:01.214090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:01.214931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:01.215722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:01.443620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:01.457909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6b8d3a96-88e5-4714-81ae-69ad829276d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.271713 Sep 19 16:16:01.458704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 160/310] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:01 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:01.949948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:01.950919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:01.959286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:01.960800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:01.961470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:01.962451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:01.963102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:01.963756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:01.991832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dffc0dc2-d281-4115-b6eb-f096618151d8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/ips" status: 200 len: 152 microversion: 2.20 time: 0.055594 Sep 19 16:16:01.993160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 151/311] 10.222.0.57 () {60 vars in 1289 bytes} [Mon Sep 19 16:16:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/ips => generated 152 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:16:02.501202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:02.529694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:02.556498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:02.558006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:02.559473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:02.561486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:02.563254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:02.564278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.155649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:03.167715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cc3ae091-bb44-4573-bac2-3a4d7488ab0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1502 microversion: 2.1 time: 0.678759 Sep 19 16:16:03.168447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 161/312] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:02 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1502 bytes in 681 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:03.193099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "04025b8f-4b10-421e-8821-8d11e78ab565"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:03.196286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:03.203052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.203635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.204143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.204859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.205308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.205842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.234468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Cannot 'attach_volume' instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 while it is in vm_state rescued Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/os-volume_attachments HTTP/1.1 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 74 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:16:03.236750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {"volumeAttachment": {"volumeId": "04025b8f-4b10-421e-8821-8d11e78ab565"}} Sep 19 16:16:03.240595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Cannot 'attach_volume' instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 while it is in vm_state rescued Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'c5d17c4e-bcff-42fa-95c7-f7df3fb97a42', 'body': {'volumeAttachment': {'volumeId': '04025b8f-4b10-421e-8821-8d11e78ab565'}}} Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:16:03.242283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 while it is in vm_state rescued Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:16:03.245112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 while it is in vm_state rescued Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 409 to user: Cannot 'attach_volume' instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 while it is in vm_state rescued {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f872241d-cf3d-4218-a9b2-ac5773764e2e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.051609 Sep 19 16:16:03.247612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 152/313] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:16:03 2022] POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/os-volume_attachments => generated 152 bytes in 53 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 19 16:16:03.252796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:03.253719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:03.262107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.262670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.263137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.264711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.264711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.265157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.369391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4984b867-28be-45b2-8280-3033dfd5b16d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/action" status: 202 len: 0 microversion: 2.1 time: 0.123069 Sep 19 16:16:03.370061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 162/314] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:16:03 2022] POST /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42/action => generated 0 bytes in 125 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:16:03.381188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:03.382249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:03.394745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.395504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.396250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.397294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:03.397855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:03.398515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:03.704390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:03.720568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2dc47a86-6f67-42f0-ac7f-2513f1e8936d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1510 microversion: 2.1 time: 0.344948 Sep 19 16:16:03.722828 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 153/315] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:03 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1510 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:04.765646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:04.773696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:04.788050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:04.789782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:04.791049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:04.792867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:04.794053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:04.795344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:04.848584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:04.850649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:04.880374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:04.882137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:04.883348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:04.885337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:04.888352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:04.890422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:04.940908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lazy-loading 'flavor' on Instance uuid 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:16:05.107388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:05.117209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-59867c8b-d84d-47ad-acac-ee2a27a54aec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1510 microversion: 2.1 time: 0.362812 Sep 19 16:16:05.118459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 163/316] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:04 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1510 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:05.701494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-1211721229", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1392065781", "security_groups": [{"name": "tempest-secgroup-smoke-1488799915"}], "block_device_mapping_v2": [{"uuid": "85efcfbb-1aab-400a-96b3-54dd1ca39d9a", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "b1a33dbe-096c-412a-bfd6-096a0168eb28"}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:05.954800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] validate_networks() for [('b1a33dbe-096c-412a-bfd6-096a0168eb28', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:16:07.010587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:16:07.011614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:16:07.012444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:16:07.013388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:16:07.014160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:16:07.036922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:07.039268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:07.040574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:07.043620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:07.051175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:07.052977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:07.134723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Getting quotas for project 2a726afa4ec84ec99a5b1bd08c95a0cd. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:16:07.139890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Getting quotas for user 9fd6c22a61be43658ae87052b867e384 and project 2a726afa4ec84ec99a5b1bd08c95a0cd. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:16:07.249188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:16:07.251467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='85efcfbb-1aab-400a-96b3-54dd1ca39d9a',volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:16:07.622269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a923bb50-5a1a-42f5-9a7d-bab8b338d8b3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.927627 Sep 19 16:16:07.638827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 164/317] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:16:05 2022] POST /compute/v2.1/servers => generated 406 bytes in 1946 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 19 16:16:07.665325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:07.668188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:07.698240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:07.699800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:07.700883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:07.702858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:07.703977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:07.704945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:07.731535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:07.972420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:07.991392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7fdd26af-c3dc-4af5-9320-8a5c0b883c3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1516 microversion: 2.1 time: 0.346945 Sep 19 16:16:07.992945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 165/318] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:07 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1516 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:08.003596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c0752714-e633-4699-a4c9-4e7b1183a0d6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:08.012536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c0752714-e633-4699-a4c9-4e7b1183a0d6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:08.221044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c0752714-e633-4699-a4c9-4e7b1183a0d6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:08.256259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c0752714-e633-4699-a4c9-4e7b1183a0d6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1095 microversion: 2.1 time: 0.261132 Sep 19 16:16:08.269638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 166/319] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:07 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1095 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:09.298611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:09.304809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:09.326681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:09.327648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:09.328846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:09.329893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:09.331129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:09.331668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:09.485064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:09.503727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fa183b82-ab4e-41b1-9ecc-735be38fbb99 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1193 microversion: 2.1 time: 0.213355 Sep 19 16:16:09.506428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 167/320] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:09 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1193 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:09.754715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6aa8a530-6f2e-4701-91a5-b9e5a7e5f689 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: .version_select at 0x7fadbfa21a60>, body: {"keypair": {"name": "tempest-keypair-1521587055", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLUrWwtu2Zj3FSh5xjJJVnxZ9DGNgR88IcW2Kg+ClvUlpAGnV1lBbonTNH4GYHj9ZUJXhDBdGn3/V5BdAkp0exJ/PaCZ3Zih7Fi7xIMDPBzuzxZrS3G8W+kT+yst4ULYew=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:09.799180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-6aa8a530-6f2e-4701-91a5-b9e5a7e5f689 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:16:09.808402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-6aa8a530-6f2e-4701-91a5-b9e5a7e5f689 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:16:09.938955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6aa8a530-6f2e-4701-91a5-b9e5a7e5f689 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.190565 Sep 19 16:16:09.946095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 168/321] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:16:09 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:10.035969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b54c841c-15f1-4605-b31e-7a5614f59a3e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: .version_select at 0x7fadbfa2b700>, body: {"keypair": {"name": "tempest-keypair-2029727403", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBOj8Pna8tLCEiduMQf0fAGOU6hAmgmSd6iE4u6EwWJW3IkYs5w5fSj42IxrCD+UGJLZh54luBsDna99YMqQDnPd/cSkY3Zq2r/CGHq9aHNwbX+h5wVJg3YrkBlQuervfTQ=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:10.102348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-b54c841c-15f1-4605-b31e-7a5614f59a3e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for project 0e35510b626e46aaae189f31b27ed663. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:16:10.106283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-703821ca-eb96-445d-87ed-c12da825b330 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action" status: 202 len: 0 microversion: 2.20 time: 5.278906 Sep 19 16:16:10.114018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-b54c841c-15f1-4605-b31e-7a5614f59a3e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for user 8cd9c727c0b444cb83497ea4f78e24d0 and project 0e35510b626e46aaae189f31b27ed663. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:16:10.121507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 154/322] 10.222.0.57 () {62 vars in 1319 bytes} [Mon Sep 19 16:16:04 2022] POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action => generated 0 bytes in 5292 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:16:10.133627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:10.138825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:10.163192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=46 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:10.166906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:10.167781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:10.168546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:10.174222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b54c841c-15f1-4605-b31e-7a5614f59a3e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.151847 Sep 19 16:16:10.175339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:10.176125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:10.176904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:10.177341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 169/323] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:16:10 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:10.535463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:10.546982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:10.552756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:10.564969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:10.569733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:10.573114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cdf30896-c25e-4144-a76d-beddba19cd2a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.448627 Sep 19 16:16:10.587312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 155/324] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:10 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 462 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:10.603646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:10.606037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:10.606890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:10.608903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:10.609544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:10.610554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:10.756972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:10.796248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fb73e701-3863-4c0b-92a9-5297c02a5d41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1193 microversion: 2.1 time: 0.261809 Sep 19 16:16:10.797355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 170/325] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:10 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1193 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:11.667471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:11.668289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:11.699850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:11.700531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:11.703254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:11.704035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:11.704529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:11.707061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:11.839057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:11.840251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:11.851213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:11.853360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:11.854637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:11.856270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:11.857391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:11.858604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:11.985177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:11.999238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:12.002594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:12.005942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fac5eb07-412a-4bcd-987d-d252e935483b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.360821 Sep 19 16:16:12.008174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:12.027205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 156/326] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:11 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:12.048642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-72b99d2b-9472-4437-a239-d4a8b156f975 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1201 microversion: 2.1 time: 0.217483 Sep 19 16:16:12.050672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 171/327] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:11 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1201 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:12.573324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1961163870", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "145a1b1d-9f41-428a-98d1-15b34e55a4ad"}], "security_groups": [{"name": "tempest-securitygroup--1720059485"}], "key_name": "tempest-keypair-1521587055", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:12.938721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] validate_networks() for [('145a1b1d-9f41-428a-98d1-15b34e55a4ad', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:16:13.054464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:13.058836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:13.104771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:13.108191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:13.112544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:13.117171 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:13.118607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:13.121208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:13.383942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:13.393980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:13.395949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:13.397650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dcd24576-af20-47b4-8ad5-335baaad59a9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.355459 Sep 19 16:16:13.398427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 172/328] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:13 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:13.404030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:13.405834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:13.419524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:13.420987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:13.421740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:13.423108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:13.423971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:13.425528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:13.537078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:13.553300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3c8285c-1904-4987-9a81-a405acd61d4f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1211 microversion: 2.1 time: 0.150102 Sep 19 16:16:13.554364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 173/329] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:13 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1211 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:13.569537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-424365101", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "32b57894-920f-4402-a64b-7ff50db77230"}], "security_groups": [{"name": "tempest-securitygroup--528690600"}], "key_name": "tempest-keypair-2029727403", "user_data": "}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:13.816734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] validate_networks() for [('32b57894-920f-4402-a64b-7ff50db77230', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:16:14.879839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:16:14.881361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:16:14.882065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:16:14.882661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:16:14.883341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:16:14.906713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:14.907562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:14.908536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:14.910563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:14.911480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:14.912151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:14.939787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:16:14.946494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:16:14.970165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:16:14.974876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:16:15.044692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:16:15.045832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:16:15.046553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:16:15.047278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:16:15.048033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:16:15.075766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d1d895d2-d8d1-4b6b-90b7-18d1f850f93e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 2.510392 Sep 19 16:16:15.078568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.079348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 157/330] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:16:12 2022] POST /compute/v2.1/servers => generated 402 bytes in 2515 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:16:15.080042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.082541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.085258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.085964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.091585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.099749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:15.100930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:15.137123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:16:15.144699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.148495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.150891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Getting quotas for user 8cd9c727c0b444cb83497ea4f78e24d0 and project 0e35510b626e46aaae189f31b27ed663. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:16:15.151893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.155083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.156131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.157732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.228780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:16:15.233771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:16:15.371749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4818d193-e7fc-40cd-bcb1-9c78c00fb797 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.816031 Sep 19 16:16:15.372960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 174/331] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:16:13 2022] POST /compute/v2.1/servers => generated 408 bytes in 1818 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:16:15.380287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:15.381409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:15.400845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.401969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.402741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.407822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:15.410765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:15.411954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:15.898907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:15.915674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=91,nova_cell1:UPDATE=6,nova_cell1:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:15.954949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:15.962477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:15.970320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9628460b-8463-4e84-9501-9a97c13aa809 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.888612 Sep 19 16:16:15.977769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 158/332] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:15.979437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:15.993796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3ea8a92e-9c42-4e2d-99a5-d0146dac140c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1279 microversion: 2.1 time: 0.618013 Sep 19 16:16:15.994760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 175/333] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:15 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1279 bytes in 621 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:16.001015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-05e66510-4f1d-4366-9894-7fa4c9edfd12 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:16.001810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c3d655ad-bdc0-4e06-a935-2d0006804c8d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:16.002270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-05e66510-4f1d-4366-9894-7fa4c9edfd12 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:16.003019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c3d655ad-bdc0-4e06-a935-2d0006804c8d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:16.302311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c3d655ad-bdc0-4e06-a935-2d0006804c8d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:16.322700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c3d655ad-bdc0-4e06-a935-2d0006804c8d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1245 microversion: 2.1 time: 0.324773 Sep 19 16:16:16.324895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 176/334] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1245 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:16.359680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-05e66510-4f1d-4366-9894-7fa4c9edfd12 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:16.371167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-05e66510-4f1d-4366-9894-7fa4c9edfd12 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1242 microversion: 2.1 time: 0.390798 Sep 19 16:16:16.375196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 159/335] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:15 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1242 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:17.052899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:17.053596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:17.056696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:17.058488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:17.071880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.075564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.076152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.078594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.080324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.081856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.082605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.088663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.089784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.091723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.092833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.097185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.529967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:17.541486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:17.543656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:17.545453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-115faa80-465f-440f-a385-b3fc9522551c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.513599 Sep 19 16:16:17.562759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 177/336] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:17 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:17.577131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:17.577754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:17.595440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.595932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.597037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.603937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.604439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.604802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.737417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:17.765968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-77a86dbe-bb6d-4009-b77f-d8270dfe1763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1241 microversion: 2.1 time: 0.198649 Sep 19 16:16:17.770523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 178/337] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1241 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:17.783049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:17.789450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:17.809415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.811904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.814846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.815839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:17.816927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:17.820108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:17.993128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:18.009924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8cffe877-6782-42be-8ae4-cf26a9cbc4fe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1294 microversion: 2.1 time: 0.233978 Sep 19 16:16:18.010617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:18.011141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 179/338] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:17 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1294 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:18.029393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:18.055236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=92,nova_api:INSERT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:18.056090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-108fc642-52a8-4610-b280-d6e550434f96 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1279 microversion: 2.1 time: 1.008847 Sep 19 16:16:18.058721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.059366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.070401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 160/339] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:17 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1279 bytes in 1017 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:18.075030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:18.264852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5c5064d-5dce-44c2-8610-006177ddd542", "tag": "d6454427-db60-426c-9f7f-ef69e803c8b5"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:18.290147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.292526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.293353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:18.312144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] Creating event network-changed:d6454427-db60-426c-9f7f-ef69e803c8b5 for instance f5c5064d-5dce-44c2-8610-006177ddd542 on hv-cinder-85837 Sep 19 16:16:18.327435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-01da8c62-25bc-46ed-8758-26b692d28ef0 req-effdd179-9173-4a5b-9811-182da6a2baa1 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070197 Sep 19 16:16:18.332031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 161/340] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:18.343351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Going to try to stop instance {{(pid=120664) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 19 16:16:18.454447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8a3a09dd-57e7-4504-88b0-d336b2242702 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action" status: 202 len: 0 microversion: 2.1 time: 0.441257 Sep 19 16:16:18.456988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 180/341] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:16:18 2022] POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action => generated 0 bytes in 445 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:16:18.466976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:18.467826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:18.478848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.482748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.483591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:18.485395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.488329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.490803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:18.591710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:18.595205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:18.629370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.631164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.631991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:18.633290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:18.634145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:18.640076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.089255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.122012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.129978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.131118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.135455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5569d3f8-295a-417f-81cd-288c0b991509 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.562351 Sep 19 16:16:19.136109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 181/342] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:19.149527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-39a5270f-a85e-4d03-aa30-01f93d49f341 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.683372 Sep 19 16:16:19.149527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 162/343] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:18 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 685 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:19.171537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:19.172448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:19.180496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.181335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.182015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.183000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.183758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.184419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.186911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:19.194833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:19.221335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.223039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.223968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.225508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.230572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.231871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.424157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.436339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.469391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7a7aa29d-58ee-4baa-88cd-fdb3972a1ca7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1297 microversion: 2.1 time: 0.283545 Sep 19 16:16:19.488671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 163/344] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1297 bytes in 341 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:19.492555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-93114605-9116-4ad1-bf7c-39dc167ab416 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1300 microversion: 2.1 time: 0.340513 Sep 19 16:16:19.493504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 182/345] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:19 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1300 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:19.511204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:19.516637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:19.531071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.538332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.542965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.542965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:19.548282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:19.548282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:19.887859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:19.904223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b99b17ad-2993-43b2-b974-f94319947147 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.413641 Sep 19 16:16:19.919028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 164/346] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:19 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:20.172851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:20.173883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:20.178500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:20.182003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:20.209204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:20.215335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:20.216720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:20.219923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:20.221741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:20.225260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:20.243547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:20.246991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:20.249165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:20.252201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:20.255069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:20.259025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:20.943026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:20.954733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:20.960763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:20.968154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ccb369f4-17c9-488a-bb58-22f5485a1448 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.799941 Sep 19 16:16:20.969093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 183/347] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 806 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:20.980460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:20.981139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:20.999221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.002100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.004724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.007643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.008398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.009039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.250484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:21.264140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fede47f2-f31f-481c-8bc2-99623bff8159 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1303 microversion: 2.1 time: 0.293742 Sep 19 16:16:21.271516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:21.280348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 184/348] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1303 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:21.296288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:21.297644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:21.306267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.306629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6ae033c3-ad9e-43d1-8f52-6b7466ac9f1f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 1.133804 Sep 19 16:16:21.306866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.309368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.310741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 165/349] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:20 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 1141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:21.312006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.312559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.313121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.331326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:21.333005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:21.368818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.370133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.373525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.376115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:21.377058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:21.379817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:21.733614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:21.740358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:21.774159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-52d1fd3e-1cec-4de3-95ec-a0152f198478 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.490819 Sep 19 16:16:21.774159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 185/350] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:21 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:22.025960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:22.030509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:22.049545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.051508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.052657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.054302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.055704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.057017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.079138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:22.094665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-eee15be4-86c6-4939-b637-e5ccd27707c7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.781248 Sep 19 16:16:22.095254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 166/351] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:21 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 784 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:22.296540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:22.305577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:22.335830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.336635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.337321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.338970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.340012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.341000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.614144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:22.640556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:22.651753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:22.666259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:22.672918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-17957526-86ed-4bb1-9a91-678ac3be6b96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.659346 Sep 19 16:16:22.677084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8e14bd6a-7196-4072-ad71-06808c8acbf8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1303 microversion: 2.1 time: 0.384971 Sep 19 16:16:22.679585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 186/352] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 669 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:22.681675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 167/353] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:22 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1303 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:22.691892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:22.694084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:22.710782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.712367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.713610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.716051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.716051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.716628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.819914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:22.821292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:22.850724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.851591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.852437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:22.853892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:22.866308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:22.867231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:23.399361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:23.417886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1ae08e98-33b9-4c55-8de5-04fa9ff0d838 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.735441 Sep 19 16:16:23.419345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 187/354] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:22 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 739 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:23.433691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:23.434770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:23.450411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:23.452299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:23.454772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:23.457167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:23.458837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:23.460452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:23.806898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:23.831346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb7e7f1b-3f43-49dd-98fc-a428b05c747c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.399869 Sep 19 16:16:23.834670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 188/355] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:23 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 413 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:23.856208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:23.857277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:23.880845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:23.882064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:23.883375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:23.892518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:23.894077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:23.894894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.388364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:24.414262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bcf8b5fe-3825-4cab-bbe7-c6bb2d50f538 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 1.601935 Sep 19 16:16:24.427624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 168/356] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:22 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 1618 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:24.439377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:24.440843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:24.464694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.465780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.467494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.468942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.469942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.471643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.567545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:24.605487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:24.614604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:24.617501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fa630a17-6dcc-4ad5-9836-be95bebb4d8e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.781227 Sep 19 16:16:24.639094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 189/357] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 803 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:24.652307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa7d958b-1b74-4d02-9a97-aae5da88ce3e", "tag": "4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:24.667161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.670784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.671511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.688137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] Creating event network-changed:4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e for instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e on hv-cinder-85837 Sep 19 16:16:24.698325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-88935cb1-5bcf-4a6d-8704-a8ab45e3ba2e req-8a898295-966c-4d04-90de-b5b1c793c1fd service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057564 Sep 19 16:16:24.699457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 190/358] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:24.710571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:24.712269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:24.729371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.731746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.734592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.735708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.736433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.737345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.842069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:24.859048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ae27333f-fe44-4b1f-be70-614c71a4ea19 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.429500 Sep 19 16:16:24.860536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 169/359] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:24 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 432 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:24.870587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:24.872004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:24.883580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.884771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.885871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.887719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:24.888853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:24.889882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:24.936124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:25.106122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:25.107378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:25.117221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2b36e394-8383-41ca-8657-d0755c17c558 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.254185 Sep 19 16:16:25.117930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 170/360] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:24 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:25.167275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:25.192191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-99cb89a1-9cb8-4dd4-9b3c-bec6a26770a4 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.490615 Sep 19 16:16:25.194693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 191/361] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:24 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:25.472290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:25.482053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:25.500543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.501402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.502171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:25.512021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.516469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.528103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:25.656041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:25.658043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:25.673333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.675116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.675963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:25.677602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.678829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.680511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:25.905897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:25.919492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-13f6c552-e622-4d09-8fcc-6a5795bd7e1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.456257 Sep 19 16:16:25.922131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 171/362] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:25 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:25.935314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:25.936196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:25.948415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.957403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.958729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:25.959751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:25.960459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:25.961141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.171577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:26.181814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:26.187765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:26.192526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9174af56-e4f3-4aef-8bbb-0e7ef1ee0eef tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.550005 Sep 19 16:16:26.208243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 192/363] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:26.220271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:26.222732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:26.230059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.230832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.231536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.232476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.233268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.233934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.320505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=24 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:26.358171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:26.366896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=8 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:26.372989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4d9362bc-7eef-4318-88cb-7bb2f5ec9092 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.449330 Sep 19 16:16:26.373615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 172/364] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:25 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 451 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:26.396553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:26.401554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:26.421640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.424053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.428984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.430135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.430903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.433151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.844356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:26.853354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-29fd0f4f-6438-4553-b985-89fd06f4caa5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.643780 Sep 19 16:16:26.865904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 193/365] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:26 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:26.959455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:26.960089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:26.981395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.981891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.982329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:26.991123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:26.991624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:26.992480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.290897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.347147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aa3df1d6-4ab7-4d5e-94a4-b9f7a175234b tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.959577 Sep 19 16:16:27.348560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 173/366] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:26 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 974 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:27.358237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:27.359319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:27.393232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.394217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.395335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.396992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.397793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.398563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.483761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.530828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-65ac1860-1a5a-406b-8ee9-2bf7aa66ece0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.565095 Sep 19 16:16:27.551524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 194/367] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:26 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:27.572754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:27.574944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:27.585623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.589444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.596880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.598335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.599362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.599995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.745013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.763960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.767359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.771619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bc6b7ad6-fab6-4e8f-8300-5f83f86b12b9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.421165 Sep 19 16:16:27.772544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 174/368] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:27.834412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:27.853787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8219c35e-1eba-4189-aa0f-55a90c706a03 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.295918 Sep 19 16:16:27.870500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 195/369] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:27 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:27.880212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:27.882752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:27.883608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:27.892648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.893453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.893955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.895115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.924743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.924743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.929943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:27.930861 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:27.931650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:27.945746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] Creating event network-changed:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:16:27.979527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-72497820-0607-4145-80a1-bcb5a9552bfc req-cc9c6f58-f851-4c36-ad63-84a79a57b392 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.117996 Sep 19 16:16:27.980319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 175/370] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:27 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:16:28.377855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:28.379359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:28.391971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.396754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.397465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.399300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.400879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.402051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.532149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:28.555913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7321467c-ef91-4ca1-ab78-c743316dfac6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.678705 Sep 19 16:16:28.557730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 196/371] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:27 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:28.583238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:28.585036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:28.599938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.601211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.602240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.605478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.607482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.608948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.645704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:28.658564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f55bcf7d-d022-41de-90fb-5d3d1fa25364 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.285304 Sep 19 16:16:28.660387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 176/372] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:28 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:28.805118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:28.805672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:28.826067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.827285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.827958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.829332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.830013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.831906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.906421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:28.919727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7102581b-3c05-4c8f-903d-e757ad383da5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.346791 Sep 19 16:16:28.920578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 197/373] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:28 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:28.944443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:28.944443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:28.949637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.951141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.955406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:28.959609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:28.960337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:28.965336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:29.071407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:29.074920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:29.083854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:29.085746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:29.087231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f84ac228-7bc6-46a6-b5b6-28d51fb33f86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.290622 Sep 19 16:16:29.088494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 177/374] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:29.247530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:29.275647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bc31cb15-5eb3-4794-be25-8db4349c6994 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.348754 Sep 19 16:16:29.277484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 198/375] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:28 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:29.586089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:29.590291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:29.610787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:29.611810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:29.612627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:29.618099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:29.618804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:29.619914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:29.690621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:29.691703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:29.704088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:29.704088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:29.704727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:29.706790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:29.707634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:29.708412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.000654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.009747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a0663cfa-9696-47c8-a0bd-81e420ce3187 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.431043 Sep 19 16:16:30.014765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.030707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 178/376] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:29 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:30.038979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:30.040018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:30.057937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.058760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.059273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.063132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.063629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.064302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.079708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-122a5f48-1b3c-454d-a410-9b2aa646bd8e tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.397373 Sep 19 16:16:30.082153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 199/377] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:29 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:30.134462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:30.135394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:30.142557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.143409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.144048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.145227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.146051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.146680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.393605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.406986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c21adf3b-4078-4ae2-b2c6-657321349d09 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.375425 Sep 19 16:16:30.407625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 179/378] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:30 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:30.422675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:30.422675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:30.423123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.423912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.426131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.427401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:30.428396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:30.429201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:30.507912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.521466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.523036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.524318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3caf7ff5-3728-4434-898b-b96840435c30 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.394851 Sep 19 16:16:30.532764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 200/379] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:30.715038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:30.731685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-39853e23-11e4-4b83-bb8e-843d7b6f5920 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.322927 Sep 19 16:16:30.747746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 180/380] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:30 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:31.036524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:31.038033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:31.064131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.065069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.065867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.067703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.068737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.069928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.124698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:31.126065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:31.169576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.180210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.182260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.188266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.190967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.191473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.381720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.390200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-67a60328-3d53-463d-8127-0df2c357465e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.360318 Sep 19 16:16:31.391665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 201/381] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:31 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:31.443065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:31.443844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:31.450558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.451330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.451770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.452761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.453222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.453691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.515153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.534472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-98ff79f1-5d8b-4e20-9052-01dbbae68ce6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.415191 Sep 19 16:16:31.536429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 181/382] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:31 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:31.566343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:31.567506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:31.580374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.580864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.581308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.582274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.582773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.587089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.683687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.699213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d7465d2a-1bcc-4c67-997a-970f3969b378 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.261539 Sep 19 16:16:31.700055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 202/383] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:31 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:31.763527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:31.764321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:31.778069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.779138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.779951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.783145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:31.790358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:31.791113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:31.833801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:31.860273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.870597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.872639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:31.873986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-33102bb8-2fc4-44b1-ba01-38bc39075e27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.314860 Sep 19 16:16:31.881868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 182/384] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:32.230684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:32.254326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3edb49be-c0fc-4a44-ab23-8f7e3e68e65f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.500212 Sep 19 16:16:32.256940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 203/385] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:31 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:32.417285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:32.419583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:32.434396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.435961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.437823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.439815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.443456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.444371 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.591638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:32.593998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:32.610244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.611188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.613222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.617313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.617920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.618552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.809531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:32.828317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5052e588-2999-4002-8775-ee303d4ceeb8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.416604 Sep 19 16:16:32.829105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 183/386] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:32 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:32.834717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:32.847320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:32.854140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.854669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.855108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.855868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:32.856304 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:32.856727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:32.926120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:32.945624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7f399257-4c9e-45e2-9193-912c693a87ac tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.359830 Sep 19 16:16:32.946983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 204/387] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:32 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:32.985860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:32.988326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:33.015552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.016639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.017393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.018719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.019500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.025856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.286801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:33.306054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-380a421c-2d0d-498b-b85d-1c7e5a2a6288 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.474741 Sep 19 16:16:33.308144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 184/388] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:32 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:33.319205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:33.320758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:33.333689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.335013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.336837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.339257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.340858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.351414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.500936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:33.523239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:33.525801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:33.529265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b38b1c59-13ba-466c-9ece-829e2693e1a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.568467 Sep 19 16:16:33.534304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 205/389] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 575 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:33.679360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:33.698334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6cc2a9e5-b9c2-46ea-a803-dbf1972d531c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.387172 Sep 19 16:16:33.699790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 185/390] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:33 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:33.872703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:33.873612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:33.884494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.885566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.886251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.887092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:33.887797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:33.888487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:33.991791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:33.998416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:34.013141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.014507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.017118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.018899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.020130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.021238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.406931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:34.416562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:34.429746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-77c653cb-ff73-4645-9806-16fa23156510 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.451985 Sep 19 16:16:34.430393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 186/391] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:33 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 455 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:34.444953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2bb278d6-2981-4273-99a9-53aa6bece80b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.574865 Sep 19 16:16:34.457120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 206/392] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:33 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 589 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:34.466633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:34.467445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:34.491275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.492258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.492804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.493542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.494012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.494513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.594427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:34.595539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:34.626052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.633275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.634329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.635626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.636427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.637151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.895544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:34.922012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f36508d5-a0bf-437b-9125-7cb43e4bf136 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.469982 Sep 19 16:16:34.925297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 187/393] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:34 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:34.937063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:34.938899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:34.967177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.968463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.969563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.971752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:34.972716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:34.974487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:34.975770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:34.993815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:34.997327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:35.000500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5136286f-e189-44ff-ab00-62c08717a47e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.418342 Sep 19 16:16:35.002940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 207/394] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 422 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:35.329702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:35.369199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-198fd241-c5c4-41ff-8afa-43b93011c3e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.441078 Sep 19 16:16:35.380807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 188/395] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:34 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:35.456853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:35.457605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:35.466648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.468547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.469260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:35.471308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.472433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.477199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:35.522587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:35.524243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:35.535808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.536715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.537532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:35.538878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.539710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.540519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:35.750176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:35.764566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a5e22ed4-400e-4a00-a392-68f8361dbe61 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.249450 Sep 19 16:16:35.765849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 189/396] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:35 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:35.906282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:35.918039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-da556716-fa61-4c0b-af9f-b282c3f58525 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.466457 Sep 19 16:16:35.919512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 208/397] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:35 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:35.956345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:35.961028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:35.981457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.988778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.989791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:35.991135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:35.992439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:35.993488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.033215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:36.033873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:36.043758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.045201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.046170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.047266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.048506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.049965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.389556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:36.419261 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-75627aee-f819-45b8-ae75-5eb3e687ff83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.463515 Sep 19 16:16:36.422073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 190/398] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:35 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 475 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:36.435497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:36.437888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:36.455047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.456290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.457347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.458923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.460002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.461093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.538641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:36.567144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:36.572119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:36.588257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-57df45cb-072f-4220-ad6f-a64c2fc23fb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.558825 Sep 19 16:16:36.604532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 209/399] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 576 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:36.740548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:36.758583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4b753e5c-1c93-4ea1-8231-e375dbd46c61 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.334280 Sep 19 16:16:36.759698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 191/400] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:36 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:36.805964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:36.806668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:36.828955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.831423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.833183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.836295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.837672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.838893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.965905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:36.972519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:36.990793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:36.991840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:36.992668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:36.993995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.010242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.011137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.085486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.112542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4859c2d5-825d-4772-a056-4779d6995ba9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.324359 Sep 19 16:16:37.113622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 210/401] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:36 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:37.285167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.300529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bbdaf64b-3249-477d-b564-28c1d74eb5de tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1723 microversion: 2.1 time: 0.343588 Sep 19 16:16:37.301247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 192/402] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:36 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1723 bytes in 346 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:37.445700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:37.446361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:37.454313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.455179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.455908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.457179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.457987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.458724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.642514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:37.643972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:37.652072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.653464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.654166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.655211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.655755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.656260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.695032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.711704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e3a5cc67-4a0f-467a-8470-733cd84c172a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.269862 Sep 19 16:16:37.713141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 211/403] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:37 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:37.786211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:37.792416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:37.829512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.847213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.848444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.850048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:37.854348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:37.860661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:37.971852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.987324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.989825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:37.991463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5efa31e1-8932-4df6-9095-28e6fb5a232b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.353293 Sep 19 16:16:37.992806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 193/404] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:38.131545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:38.144445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-909e4f1c-4db4-4a6f-8d81-a8155a9fedc2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.365383 Sep 19 16:16:38.153613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:38.155543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:38.159396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 212/405] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:37 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:38.169680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.171528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.172626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.174519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.175733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.176849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.214217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:38.340135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:38.341667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:38.351103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.351786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.352505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.353408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.354196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.355106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.394931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:38.403395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1e092364-1141-4b04-b55a-eceee7872a79 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.257541 Sep 19 16:16:38.404529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 194/406] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:38 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:38.576081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:38.593191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d26288b8-091b-480b-aec8-a76833d6df59 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1700 microversion: 2.1 time: 0.258098 Sep 19 16:16:38.607421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 213/407] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:38 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1700 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:38.630906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:38.642430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.644303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.645132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.756871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:38.759382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Going to try to start instance {{(pid=120663) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 19 16:16:38.771496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:38.790039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.791140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.792208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.793581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.794918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.796235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.821711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-01628ac9-a6de-46b7-9324-62cfd340e703 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action" status: 202 len: 0 microversion: 2.1 time: 0.199737 Sep 19 16:16:38.824263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 195/408] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:16:38 2022] POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:16:38.843616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:38.844718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:38.848157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:38.859434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.860265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.861041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:38.862847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:38.863650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:38.864377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.080098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.094970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f50a3462-8b9a-4c2b-b1d1-b3ebb0499921 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.357769 Sep 19 16:16:39.110458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 214/409] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:38 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:39.110458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:39.110912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:39.117838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.118788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.119106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.120057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.121028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.121730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.122364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.134908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-96e1f99a-d81e-49de-8242-d6e4cc0a5ede tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.305202 Sep 19 16:16:39.135774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 196/410] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:38 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:39.187419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:39.188599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:39.200105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.201333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.202084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.203340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.204072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.204740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.312174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.329122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.333151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.335031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f524e46c-8ec6-48cb-9cd8-70cfa8f3f882 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.230100 Sep 19 16:16:39.336076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 215/411] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:39.418917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.433838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-99e4a2c6-095f-4f6a-9ced-2020c0aecc65 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.255182 Sep 19 16:16:39.434464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 197/412] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:39 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:39.445426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:39.446719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:39.454519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.455083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.457226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.458260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:39.459121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:39.460027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:39.675809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:39.689533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5526b333-dacd-464b-8263-e1a02fef7544 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.249807 Sep 19 16:16:39.690456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 216/413] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:39 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:40.134339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:40.135434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:40.143736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.144221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.144658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.145378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.145831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.146252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.176444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:40.177185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:40.183749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.184247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.184686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.185402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.185856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.186519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.344360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.365130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-320e902d-e7c3-41d1-a25c-089da2857563 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.234511 Sep 19 16:16:40.366227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 198/414] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:40 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:40.374713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.383938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:40.384529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-97aa316c-85a5-4629-840b-bbe1f1fcb071 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.212259 Sep 19 16:16:40.385111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:40.385413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 217/415] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:40 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:40.398355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.398355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.398355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.398942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.399224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.400038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.464927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:40.466021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:40.474726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.475375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.476208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.477343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.478185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.478938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.564006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.573306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.574872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.576105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ab85ac60-1002-4a57-ba33-64448f104602 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.200284 Sep 19 16:16:40.576713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 199/416] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:40.656470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.664578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9ca42076-753d-4684-866f-d91e2883b654 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.204242 Sep 19 16:16:40.665176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 218/417] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:40 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:40.719427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:40.720311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:40.727560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.728201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.728640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.729364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:40.729812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:40.730478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:40.925468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:40.934572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7438953b-8a39-43b0-bd55-bddfbff88fe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.218992 Sep 19 16:16:40.935533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 200/418] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:40 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:41.391462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:41.392112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:41.399175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.399668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.400089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.400803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.401258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.401684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.418722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:41.419508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:41.427689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.428270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.428785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.429839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.430581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.431134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.582669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.586610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:41.591832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f96856fd-82b9-45b4-af1f-cdcb0539a5d3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.204199 Sep 19 16:16:41.592465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 219/419] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:41 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:41.605130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:41.606142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:41.619202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.619850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.620331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.621181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.621487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.621963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.622543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.636599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8b9dca6d-0f2e-48f4-b808-0cbcfd65f1a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.223578 Sep 19 16:16:41.637202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 201/420] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:41 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:41.692493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:41.693315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:41.700808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.701387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.701902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.702844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.703602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.704293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.807654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.821225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.823040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.824384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-02ce595a-77de-4167-aae8-140b7096ad81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.223868 Sep 19 16:16:41.825149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 220/421] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:41.892797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:41.902125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-feedb2d8-c15a-4924-b908-76e85f75b01f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.215541 Sep 19 16:16:41.914464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 202/422] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:41 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:41.977569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:41.979139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:41.991423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.992366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.993607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:41.995138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:41.995999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:41.996821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:42.248191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:42.264247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8853ca27-f786-43f3-8d11-18aac1e2ecb4 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.293521 Sep 19 16:16:42.273798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 221/423] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:41 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:42.630741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:42.630741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:42.643517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:42.644376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:42.645165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:42.646481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:42.647313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:42.648092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:42.668521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:42.669562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:42.693938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:42.694871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:42.700687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:42.702848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:42.703982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:42.706829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.002320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.009585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.013318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ebe808b1-80d9-40a7-9c4b-f1a5adc968e8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.401426 Sep 19 16:16:43.014252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 203/424] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:42 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:43.019968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:43.020582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:43.029041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.029800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.030443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.031190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.031855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.032467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.055639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-28215f00-700b-4791-99f4-7875d86e062d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.394143 Sep 19 16:16:43.056686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 222/425] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:42 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:43.066583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:43.068004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:43.079646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.081222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.082012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.083332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.084155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.084941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.213279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.224475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.227240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.228678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2ef464e2-bd16-4aff-bab2-f39cd783c356 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.213231 Sep 19 16:16:43.229376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 204/426] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:43.295908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:43.298540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:43.318256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.319551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.320552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.322107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.322978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.323754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.359715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.373968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e09f20f7-17cb-4fe7-8469-48a21a73dd79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.315356 Sep 19 16:16:43.380545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 223/427] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:43 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:43.566459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "name": "network-vif-plugged", "status": "completed", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:43.577578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:43.578281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:43.579208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:43.595392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] Creating event network-vif-plugged:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:16:43.601970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-9a53e353-8484-4e0e-b243-d42cb3cb866f req-916471f4-9fb7-4076-b09e-b4a78d9878df service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.040122 Sep 19 16:16:43.603647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 224/428] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:43 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:16:43.608808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:43.620070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4c690209-aa5a-4b54-a2af-ce64926636ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.335320 Sep 19 16:16:43.621473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 205/429] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:43 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:44.059619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:44.060534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:44.068378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.069011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.069543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.070395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.070987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.071495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.091711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:44.092836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:44.112452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.113664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.114695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.115987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.116847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.117653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.243049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.269999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5058edd1-e39c-46d5-941f-d62a6e55e22c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.214681 Sep 19 16:16:44.271098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 225/430] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:44 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:44.279279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:44.280283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:44.291046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.291909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.292563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.293612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.294301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.295003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.319720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.336931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cfa20aa2-c1c1-47cb-8697-602457dab1b2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.252726 Sep 19 16:16:44.338128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 206/431] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:44 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:44.421963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:44.423279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:44.434607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.435494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.436265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.437639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.438447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.440247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.477906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.490124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.491906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.493311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1610e7f9-2fe6-4ecc-9045-bec265218bf2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1725 microversion: 2.20 time: 0.220824 Sep 19 16:16:44.493989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 226/432] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1725 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:44.630567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.641179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f1aee6a1-d94b-4b9f-964c-74f6f5358a52 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.226978 Sep 19 16:16:44.642620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 207/433] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:44 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:44.648836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:44.649845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:44.659301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.660050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.660784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.662101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:44.662838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:44.663613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:44.856077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:44.865753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4a91dd1b-995c-44d2-9cc8-fc8c79f6ac24 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.223313 Sep 19 16:16:44.866842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 227/434] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:44 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:45.302756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:45.306977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:45.333752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.334816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.336993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.348442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.349132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.349714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.362143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:45.363107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:45.372457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.374879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.375588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.387808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.391490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.393729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.848167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:45.866569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc64e9d2-7ec0-4abe-be69-b0011b161dba tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.506852 Sep 19 16:16:45.866569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 228/435] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:45 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 508 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:45.902771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:45.904102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:45.914225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:45.927069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.927943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.928446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6d971d64-a04b-46ef-b88c-74709a8a8117 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.633692 Sep 19 16:16:45.928766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.930126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.931020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.938339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.942552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 208/436] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:45 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 649 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:45.948384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:45.949429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:45.957409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.958315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.959015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:45.959968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:45.961454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:45.962209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.325949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:46.338416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:46.340175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:46.341792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-245a7c3c-60a8-4f65-8744-090e69f392f4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1746 microversion: 2.20 time: 0.456236 Sep 19 16:16:46.342891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 229/437] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1746 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:46.349295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:46.350603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:46.354834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:46.360896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.361800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.362500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.363804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.364454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.365234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.439570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:46.448232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-23a78a56-bba6-4f19-849b-bfc0542f566f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.503856 Sep 19 16:16:46.449163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 209/438] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:45 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:46.589839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:46.599612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3615035-e94f-4ea2-8aa9-eb4eb74cfa6c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.255282 Sep 19 16:16:46.600755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 230/439] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:46 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:46.650821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5c5064d-5dce-44c2-8610-006177ddd542", "name": "network-vif-plugged", "status": "completed", "tag": "d6454427-db60-426c-9f7f-ef69e803c8b5"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:46.669000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.669519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.670075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.677234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:46.686098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] Creating event network-vif-plugged:d6454427-db60-426c-9f7f-ef69e803c8b5 for instance f5c5064d-5dce-44c2-8610-006177ddd542 on hv-cinder-85837 Sep 19 16:16:46.694089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-058fffba-b2c0-405b-89f1-d7014e77b9f6 req-e835c4ff-daa4-4393-aa75-6985559e3095 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050128 Sep 19 16:16:46.694863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 210/440] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:46 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:46.903410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:46.904637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:46.916492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.917343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.918103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.919397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.920200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.921202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.961431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:46.962215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:46.969419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.969912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.970348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:46.971778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:46.972395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:46.972959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.137498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.138551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.149584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-04e38cc9-dcc1-4670-bdcc-1812c9870ea0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.253049 Sep 19 16:16:47.149869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad007de1-48e2-4908-8249-d9993a0ba857 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.193556 Sep 19 16:16:47.150833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 211/441] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:46 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:47.151034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 231/442] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:46 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 255 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:47.368716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:47.369419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:47.376127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.376615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.377041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.377734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.378192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.378635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.475463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:47.476075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:47.482332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.482852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.483294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.483996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.484451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.484876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.530869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.544508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.547773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.549920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0466915c-1bc0-463e-b0ee-ff13a764e480 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1746 microversion: 2.20 time: 0.184989 Sep 19 16:16:47.551013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 212/443] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1746 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:47.624322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:47.625190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:47.634403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.635159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.635764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.636679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:47.637320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:47.637980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:47.642141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.654142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-80511464-7881-4ef6-937e-2c7d37b5cbfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.182045 Sep 19 16:16:47.656578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 232/444] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:47 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:47.796820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:47.806017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-efbd0bcc-73b0-4b19-bf67-83c71999d76d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.186138 Sep 19 16:16:47.806675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 213/445] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:47 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:48.175330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:48.175967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:48.176268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:48.176892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:48.183512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.183803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.184011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.184388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.184612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.185020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.185197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.185613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.185996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.186175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.186689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.187251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.344872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.353153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2e714278-324d-4b86-ba95-804eb16aa8dd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.181202 Sep 19 16:16:48.353736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 214/446] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:48 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:48.354613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.362526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-564c6bf6-f6ec-467f-b7ac-cd1ba5982e9b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.191177 Sep 19 16:16:48.363099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 233/447] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:48 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:48.577493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:48.578020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:48.584741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.585223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.585655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.586355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.586828 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.587252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.677843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:48.678525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:48.684843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.685332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.685800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.686559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.687023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.687451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.734510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.745584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.747115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.748321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-db49cd22-1d44-46c3-9212-750bd72c5e62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1746 microversion: 2.20 time: 0.174910 Sep 19 16:16:48.748891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 215/448] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1746 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:48.829871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:48.830479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:48.836881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.837369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.837863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.838602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:48.839055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:48.839475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:48.848143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:48.861711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-528ffff7-f287-4533-b28a-e4abc7706a86 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.187327 Sep 19 16:16:48.862711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 234/449] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:48 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:48.991880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:49.006437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4cce31d7-76a4-4a5c-ba30-99be8ba54803 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.180351 Sep 19 16:16:49.011621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 216/450] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:48 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:49.381220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:49.382315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:49.393250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:49.395147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:49.395347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.396176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.396980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:49.398643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.399723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.400692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:49.410559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.411927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.412967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:49.418161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.422361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.423624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:49.675959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:49.699527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-db2b4b67-6544-4bfe-9a32-85eee664a667 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.325765 Sep 19 16:16:49.700539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 235/451] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:49 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:49.775946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:49.776991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:49.788920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.791780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.794116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:49.798834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:49.800331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:49.800903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.199741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.232992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5137c097-ebab-44d0-83eb-d577dc191e39 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.850398 Sep 19 16:16:50.234716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 217/452] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:49 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 854 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:50.243618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.267208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.268864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:50.274074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:50.283393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.285526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9170fac8-4257-485d-9992-4c2bbdc73cb2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1746 microversion: 2.20 time: 0.513932 Sep 19 16:16:50.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 236/453] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1746 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:50.300908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.303736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.304934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.306301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.309824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.310583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.312933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:50.319117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:50.350064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.351026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.351521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.352285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.352791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.353339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.359581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:50.604797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.619005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.622433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1e8abbee-c799-4024-9cbb-b1c45185abee tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.385320 Sep 19 16:16:50.623872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 218/454] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:50 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 388 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:50.643974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3361c495-ef6e-4dac-b6fb-aa3088d3deb9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.346472 Sep 19 16:16:50.644870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 237/455] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:50 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:50.734229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:50.735241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:50.743121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.744063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.745044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.746224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:50.747166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:50.747879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:50.945597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:50.956246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b1eaaaec-5f60-4732-aacf-866d492147cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.234050 Sep 19 16:16:50.956823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 219/456] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:50 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:51.289615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:51.290597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:51.298211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.300168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.310188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.310922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.311101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.311606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.333409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:51.334065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:51.346193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.348605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.348605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.348605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.350451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.350975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.606855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:51.627632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:51.629763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:51.631466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-03465c3f-8c45-4d6e-888f-045a9c7329c9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.301685 Sep 19 16:16:51.647172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 220/457] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:51.652293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:51.652924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:51.669592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.670095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.670829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.671631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.672079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.672512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.682551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:51.703918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-afeeb50d-66f4-423c-838b-741c2da94940 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.418523 Sep 19 16:16:51.705483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 238/458] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:51 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 421 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:51.712059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:51.718319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:51.737082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.737082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.740432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.741344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:51.741916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:51.742388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:51.989835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.002690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2af87122-05fe-4cea-980e-0e7a777f0bdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.357378 Sep 19 16:16:52.003736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 221/459] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:51 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:52.014814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa7d958b-1b74-4d02-9a97-aae5da88ce3e", "name": "network-vif-plugged", "status": "completed", "tag": "4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:52.025226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.025936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.026513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.041668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] Creating event network-vif-plugged:4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e for instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e on hv-cinder-85837 Sep 19 16:16:52.047837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-be38aaf1-1071-4b25-884b-db081fb436d8 req-7a4b23be-97dd-408b-852f-e6fa11c6960a service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042454 Sep 19 16:16:52.049336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 222/460] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:52 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:52.056134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:52.057085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:52.064436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.065254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.065850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.066802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.067423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.068015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.152660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.165157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2b0824fc-7b2f-4d78-aba8-fd27e4558bdd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1267 microversion: 2.1 time: 0.457895 Sep 19 16:16:52.166258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 239/461] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:51 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1267 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:52.349570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.361939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dd2179a7-9021-4465-a98c-3a6d3cf460d9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1709 microversion: 2.1 time: 0.311174 Sep 19 16:16:52.362853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 223/462] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:52 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1709 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:52.664394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:52.665236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:52.672612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.674017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.674587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.675430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.676181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.676751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.743739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:52.746209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:52.772725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.773499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.775714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.776745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:52.778296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:52.779786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:52.944957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.965539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.968010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:52.970721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6977fff5-a32a-4e09-a862-08bd7a03eecf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.311880 Sep 19 16:16:53.021924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 240/463] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 364 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:53.070080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:53.071676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:53.086665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:53.087590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.090218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.093669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.095272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.097461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.115096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed2e43d6-b04e-4645-bea4-949fce5d6dfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.379134 Sep 19 16:16:53.122712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 224/464] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:52 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:53.206049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:53.207269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:53.226272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.227949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.228897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.229998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.230881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.232031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.303829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:53.313451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-20243281-953a-4154-b714-bd7f0fffdb80 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.253521 Sep 19 16:16:53.324666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 241/465] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:53 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:53.428995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:53.433910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:53.453207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.457056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.458949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.462114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.464455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.466558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.780624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:53.810756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-51364910-e6dc-4212-899a-d319a64da6e5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1713 microversion: 2.1 time: 0.408619 Sep 19 16:16:53.812627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 242/466] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:53 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1713 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:53.856875 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:53.866612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3714ccf8-8b6f-41cb-bbdc-7acff53ecc8a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1454 microversion: 2.1 time: 0.668624 Sep 19 16:16:53.874284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 225/467] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1454 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:53.920163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:53.921909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:53.934878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.936080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.938453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:53.940721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:53.942166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:53.942916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.025430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:54.026420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:54.040332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.043874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.044331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.045075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.047102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.048076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.346893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:54.374516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f523d54c-6906-4aa0-971b-e8dc027dd471 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1454 microversion: 2.1 time: 0.461814 Sep 19 16:16:54.376401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 243/468] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1454 bytes in 466 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:54.441072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:54.442584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:54.475683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.476963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.477987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.481391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.483669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.485379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.536274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:54.561145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:54.565555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:54.570721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ffb56205-3c49-4ad0-afb5-e4811e339f6c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.550673 Sep 19 16:16:54.573150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 226/469] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:16:54.585096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:54.587296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:54.600475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.601695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.602983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:54.605493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:54.607108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:54.608128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:55.097502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:55.133454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-94375e1d-3a12-42a2-9775-2c489e25ddbe tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1368 microversion: 2.1 time: 0.754581 Sep 19 16:16:55.138864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 244/470] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:54 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1368 bytes in 761 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:55.288988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:55.315914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d90af312-9150-4fbe-bd1d-5ab4a8d5e5a7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.739928 Sep 19 16:16:55.317809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 227/471] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:54 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 744 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:55.637746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:55.639732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:55.651864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:55.654326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:55.655894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:55.657457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:55.658327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:55.659404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:56.018303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:56.030995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:56.036485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:56.039816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58cb0613-18ac-4591-baf5-ad71865036b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.413134 Sep 19 16:16:56.042221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 245/472] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:56.214532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:56.215398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:56.242606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:56.248690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:56.254950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:56.256276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:56.259287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:56.260034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:56.293133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:16:56.349592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:56.353970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:56.379011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:56.383073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:56.389417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:56.392574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:56.396768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:56.400690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:56.820042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:56.842635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5a808ec3-24c1-4810-8ae4-31449fff2f43 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1558 microversion: 2.1 time: 0.661241 Sep 19 16:16:56.866523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 228/473] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:56 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1558 bytes in 671 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:56.929710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "name": "network-vif-plugged", "status": "completed", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:16:56.973325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:56.977275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:56.980832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:57.039963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] Creating event network-vif-plugged:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:16:57.123214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:57.148557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-1406ab49-e5d5-4250-828d-457c9c795e95 req-b7edcd67-b234-4c0a-853e-0d9588120d24 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.293988 Sep 19 16:16:57.150478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 229/474] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:16:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:57.161632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3d91b196-75cf-48ec-b02c-ee7b58da555c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.820762 Sep 19 16:16:57.176699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:57.177680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:57.213113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:57.214773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:57.215752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:57.218366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:57.221207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 246/475] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:56 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 882 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:16:57.222553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:57.225084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:57.249828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:57.258658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:57.281003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:57.286543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:57.289767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:57.295785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:57.300590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:57.303892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:57.680283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5af304dd-1557-4a37-9af5-082b2f7dadfc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.527278 Sep 19 16:16:57.681037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 230/476] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:16:57 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/os-interface => generated 285 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:16:57.805161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:57.840123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:57.843905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:57.852989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aa1d8c08-9b9f-4b04-beff-8ca024c465af tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.626841 Sep 19 16:16:57.874595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 247/477] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 653 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:58.214584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:58.215488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:58.234311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:58.235262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:58.236437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:58.237360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:58.237988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:58.238648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:58.885508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:58.899949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e6dcb26f-305f-4f38-a47d-079523e8eed0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.690832 Sep 19 16:16:58.901488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 231/478] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:58 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 693 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:16:58.903477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:58.905356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:58.920150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:58.921480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:58.922760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:58.925379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:58.934875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:58.937759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:59.297046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:59.310760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:59.312798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:16:59.315368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d7dbd39e-aeb6-4fe0-8e87-f5d2a9c1d794 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.419132 Sep 19 16:16:59.336588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 248/479] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:16:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:16:59.949389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:16:59.955542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:16:59.965905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:59.975530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:59.978365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:16:59.979791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:16:59.980794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:16:59.981791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:00.351228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:00.352330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:00.378311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:00.379184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:00.394389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.015s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:00.396290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:00.400840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:00.404166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:00.468970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:00.480683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c026e91f-e6d7-4ca3-980b-ec801a7e6241 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1370 microversion: 2.1 time: 0.542236 Sep 19 16:17:00.485102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 232/480] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:16:59 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1370 bytes in 548 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:00.974299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:01.019594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:01.038933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:01.049919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e6ef54bb-24b3-487e-bf8f-54ef6dd5b65a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.698801 Sep 19 16:17:01.056416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 249/481] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 714 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:01.515845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:01.517890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:01.542623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:01.548999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:01.550486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:01.551560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:01.552293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:01.555871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.006116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fa7d958b-1b74-4d02-9a97-aae5da88ce3e", "tag": "4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:02.014868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:02.036981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:02.037484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:02.037931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.052185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9013eaeb-9cf1-439e-bcba-9455734763fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1564 microversion: 2.1 time: 0.541220 Sep 19 16:17:02.069069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 233/482] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:01 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1564 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:02.098872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:02.100108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:02.116620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:02.117433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] Creating event network-changed:4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e for instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e on hv-cinder-85837 Sep 19 16:17:02.118256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:02.120328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.124879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:02.127702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:02.128701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.136005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-cc3b26ee-127a-4fd9-8fed-08d9974b63ae req-5ec08414-9f22-41f3-b5fa-c3846fc48a87 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.133517 Sep 19 16:17:02.136652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 250/483] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:02 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:02.152819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:02.156408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:02.168005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:02.172265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:02.174085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.175772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:02.178612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:02.180317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:02.517694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:02.531198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:02.533848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:02.535335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-70c754ef-c7da-426f-96af-a3305c6e4e64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.391619 Sep 19 16:17:02.536088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 251/484] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 393 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:02.557506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5e8f1fe8-3d32-460e-a964-fe6fe72ecf7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.465070 Sep 19 16:17:02.559023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 234/485] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:17:02 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-interface => generated 284 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:03.633685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:03.634626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:03.659583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:03.660221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:03.660737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:03.661533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:03.662063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:03.662600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:04.220568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:04.247047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:04.260590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:04.261863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9df8690d-b62d-48df-98ca-a37e584450e3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.637011 Sep 19 16:17:04.267048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 252/486] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 643 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:04.631288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "f5c5064d-5dce-44c2-8610-006177ddd542", "tag": "d6454427-db60-426c-9f7f-ef69e803c8b5"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:04.645887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:04.649596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:04.650315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:04.681168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] Creating event network-changed:d6454427-db60-426c-9f7f-ef69e803c8b5 for instance f5c5064d-5dce-44c2-8610-006177ddd542 on hv-cinder-85837 Sep 19 16:17:04.703919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-cb5176e0-a34b-476d-83b0-481fb5dab988 req-93c61d1b-0259-45bc-9b19-6b67e8c974ee service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.084110 Sep 19 16:17:04.705158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 235/487] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:17:05.324474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:05.325418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:05.339703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:05.340977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:05.346021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:05.346996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:05.349448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:05.350247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:05.768925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:05.797391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:05.798759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:05.799214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:05.832122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] Creating event network-changed:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:17:05.847549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-a40ba632-9b18-47d2-aa4b-1247277706af req-09f3b869-944b-483d-ba2d-5d1f27c07a6e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.083225 Sep 19 16:17:05.857799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 236/488] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:05 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:06.330983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:06.393213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:06.398473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:06.399760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bbc547d0-b3c5-4897-8aa2-b3034731d826 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 1.079781 Sep 19 16:17:06.402864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 253/489] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 1083 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:07.449936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:07.450847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:07.453895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:17:07.454121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:17:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (ip 10.222.0.57) !!! Sep 19 16:17:07.493080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:07.493987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:07.494824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:07.496148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:07.507072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:07.508273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:07.928490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:07.952924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:07.956557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:07.974416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b747a2c0-e976-487e-ab26-a36f2d932c26 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.529422 Sep 19 16:17:07.978198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 237/490] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 534 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:09.038023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:09.044114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:09.086795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:09.100075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:09.100874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:09.102142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:09.111006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:09.111934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:09.246257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:09.880121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:09.929516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:09.932689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:09.951421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d14466a5-839f-4d8b-acf2-d4a97c6080fa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.904404 Sep 19 16:17:09.953758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 254/491] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:09 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 924 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:11.021838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:11.022907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:11.024933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=84 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:11.038820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:11.040499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:11.045615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:11.050149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:11.052151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:11.054040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:11.483840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-391549405", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "networks": [{"uuid": "9949e5a2-e261-45a4-86fa-cce99f0752df"}]}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:11.533836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:11.568706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] validate_networks() for [('9949e5a2-e261-45a4-86fa-cce99f0752df', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:17:11.572447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:11.575098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:11.580911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0a0064d1-47ce-402b-91a3-957cf4491072 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.564831 Sep 19 16:17:11.583699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 238/492] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:11 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:12.351262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:17:12.351909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:17:12.352576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:17:12.353296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:17:12.354032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:17:12.370948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:12.371637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:12.372396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:12.373298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:12.374010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:12.374941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:12.417448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Getting quotas for project 5fc6461a081c479fa20c9fb15a2b1575. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:17:12.424158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Getting quotas for user e1c0b2cfaa9f485f9116e012c62120fb and project 5fc6461a081c479fa20c9fb15a2b1575. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:17:12.442136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:17:12.445158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:17:12.554759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f33ef1a8-9cf8-4034-9cc1-0f11116246c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.076046 Sep 19 16:17:12.576348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 255/493] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:17:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 1099 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:17:12.590756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3682623f-c2c0-4be9-817f-bda50e91f453 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:12.591687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3682623f-c2c0-4be9-817f-bda50e91f453 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:12.646869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:12.648149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:12.667556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:12.668755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:12.669788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:12.673944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:12.675215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:12.676289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:12.849083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3682623f-c2c0-4be9-817f-bda50e91f453 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:12.921975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3682623f-c2c0-4be9-817f-bda50e91f453 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1218 microversion: 2.1 time: 0.334107 Sep 19 16:17:12.924922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 239/494] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:12 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1218 bytes in 342 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:12.997068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:13.012861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:13.015383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:13.017490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a4823f3d-f94d-465b-82a7-2b724e4ead12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1741 microversion: 2.20 time: 0.384007 Sep 19 16:17:13.018719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 256/495] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1741 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:13.957029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:13.958611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:13.974412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:13.975879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:13.977444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:13.979459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:13.981017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:13.982286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:14.058037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:14.059598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:14.077391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:14.078294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:14.081916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:14.083370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:14.084243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:14.085054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:14.096556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:14.118224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0dc44d65-c85a-45b4-99ab-fdb348c29d33 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1270 microversion: 2.1 time: 0.170419 Sep 19 16:17:14.132906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 240/496] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:13 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1270 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:14.342635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:14.361773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:14.365052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:14.368431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0a4d7a8c-4ba7-4f8c-aefb-c5055ade5373 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1723 microversion: 2.20 time: 0.319218 Sep 19 16:17:14.369399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 257/497] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1723 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:15.163677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:15.164666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:15.174992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:15.175765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:15.176405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:15.177547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:15.178284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:15.178957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:15.333566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:15.341295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a6cf63e8-60ea-42b8-bb72-ee26ea6748c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1270 microversion: 2.1 time: 0.182133 Sep 19 16:17:15.341983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 241/498] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:15 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1270 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:15.417305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:15.417993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:15.426272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:15.426879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:15.427676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:15.428632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:15.429128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:15.429693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:15.635050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:15.651822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:15.654908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:15.656263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-51dc0649-d2a4-46d8-8bd8-74868b091018 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1723 microversion: 2.20 time: 0.243589 Sep 19 16:17:15.657082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 258/499] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1723 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:16.384782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:16.385950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:16.427497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "name": "network-vif-unplugged", "status": "completed", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:16.430966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:16.431712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:16.432165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:16.440354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=69,nova_api:INSERT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:16.441655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:16.442342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:16.443054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:16.458908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:16.471361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:16.472063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:16.499357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] Creating event network-vif-unplugged:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:17:16.511478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-9c5e1f9f-f46f-40c3-8430-ba3117bb9b66 req-e80b596b-3f13-42bf-a417-3db95557e517 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.088592 Sep 19 16:17:16.518139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 259/500] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:16 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:17:16.575414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:16.591149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f71dd1c4-9b11-47c3-a2bc-ac0ffc717386 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1270 microversion: 2.1 time: 0.213745 Sep 19 16:17:16.592400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 242/501] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:16 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1270 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:16.684401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:16.685147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:16.700584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:16.701050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:16.701563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:16.702246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:16.703037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:16.703438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:17.633272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:17.634486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:17.662141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:17.666930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:17.667685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:17.670317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:17.671983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:17.673154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:17.995075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:18.011594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:18.013295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:18.015128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-898edd00-2a24-4f19-8747-af25d8239fee tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1726 microversion: 2.20 time: 1.334608 Sep 19 16:17:18.016109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 260/502] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1726 bytes in 1337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:18.048153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:18.058946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-52baa99d-2fc6-4bd7-bd58-e9903ca2a75e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1276 microversion: 2.1 time: 0.429621 Sep 19 16:17:18.063885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "41f99edb-cead-4e63-be1b-662fbf00b32c"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:18.075645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:18.078433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 243/503] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1276 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:18.104542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:18.105564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:18.106572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:18.114776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:18.116435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:18.117570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.095555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:19.104985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:19.170779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-474ee792-c425-45b5-9de9-c930e8f5f83c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 1.118836 Sep 19 16:17:19.183646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.184704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.185522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.186889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.187513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 261/504] 10.222.0.57 () {62 vars in 1364 bytes} [Mon Sep 19 16:17:18 2022] POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 188 bytes in 1144 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:17:19.193167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.194854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.600075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:19.601583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:19.614996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.616208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.617236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.619733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.620887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.621942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.700143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120664) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 19 16:17:19.818010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:19.833055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a087ea02-f18c-4c0b-a840-e3e38da93e63 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.749561 Sep 19 16:17:19.834795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 244/505] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:19 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 753 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:19.905909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-544975cf-ef3b-4021-bf3f-4e4c0f09af6a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action" status: 202 len: 0 microversion: 2.20 time: 0.315305 Sep 19 16:17:19.907178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 262/506] 10.222.0.57 () {62 vars in 1319 bytes} [Mon Sep 19 16:17:19 2022] POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action => generated 0 bytes in 319 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:17:19.921793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:19.922338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:19.947720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.951611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.952111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:19.955234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:19.955864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:19.958348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:20.294868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:20.306016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:20.307683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:20.308898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-97ae94da-7735-484c-b9e3-591837649b86 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1752 microversion: 2.20 time: 0.396311 Sep 19 16:17:20.309468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 245/507] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1752 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:20.882277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:20.885673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:20.916568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:20.917946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:20.919997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:20.927731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:20.930408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:20.935531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:21.339034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:21.340415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:21.351385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:21.351891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:21.352331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:21.353058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:21.353515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:21.353945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:21.494143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:21.505647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-521f0c6a-279f-4d6b-b557-dff83ca6c6f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.629504 Sep 19 16:17:21.522473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 263/508] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 635 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:21.599817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f", "tag": "2f27510e-988b-4adf-8776-39c67b3541ce"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:21.652494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:21.655161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:21.656213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:21.687379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] Creating event network-changed:2f27510e-988b-4adf-8776-39c67b3541ce for instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f on hv-cinder-85837 Sep 19 16:17:21.709762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-69191766-8aed-440a-a672-6a7819662a33 req-8c3fb6ba-3e66-4698-bca6-784248ac43d4 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.115584 Sep 19 16:17:21.710409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 264/509] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:17:21.787643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:21.822206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:21.828425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:21.832103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-984f2f46-850b-42bc-b0d3-3e15bfcd1b9f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1752 microversion: 2.20 time: 0.496953 Sep 19 16:17:21.835116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 246/510] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1752 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:22.527652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:22.529974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=6 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:22.539836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:22.541355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:22.555721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:22.556568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:22.557694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:22.558990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:22.559817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:22.560860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:22.833370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:22.852314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d0ac9dd2-f01d-4017-a5f4-f938305e1570 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.325584 Sep 19 16:17:22.853823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 265/511] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:22.885311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:22.886278 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:22.890010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:22.898584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:22.899605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:22.900384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:22.901659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:22.902519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:22.903310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:23.263132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:23.280322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:23.284587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:23.286953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-282f3888-1ed9-4c22-8d03-f5e88aa79b29 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1752 microversion: 2.20 time: 0.405272 Sep 19 16:17:23.288027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 247/512] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1752 bytes in 408 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:23.898177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:23.902721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:23.923525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:23.925048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:23.925886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:23.931617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:23.932826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:23.933302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:24.320786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:24.323994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:24.359872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:24.361217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:24.361953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:24.371813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:24.372747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:24.373539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:24.392658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:24.409022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:24.411720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-058f722a-c780-41a7-8287-0af954cadbf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.537175 Sep 19 16:17:24.442406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 266/513] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:23 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:24.661078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:24.673301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:24.675664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:24.679079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-350797db-ae76-4cd5-a823-2bc2c6aeb67f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1754 microversion: 2.20 time: 0.367572 Sep 19 16:17:24.679975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 248/514] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:24 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1754 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:25.469967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:25.473458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:25.506475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:25.507403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:25.508147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:25.509473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:25.510298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:25.511054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:25.713952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:25.714630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:25.739032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:25.739885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:25.740649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:25.745084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:25.745787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:25.753731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:25.905188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:25.914940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b73cfa2f-6db7-473d-b62d-bb08ea3f4120 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.462447 Sep 19 16:17:25.926859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 267/515] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:25 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 476 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:25.982935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:25.995922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:25.998144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:26.000107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-20bd612d-a5e9-4fb5-af08-665651a2a243 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.296201 Sep 19 16:17:26.001541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 249/516] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:26.536135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f", "name": "network-vif-plugged", "status": "completed", "tag": "2f27510e-988b-4adf-8776-39c67b3541ce"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:26.562566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:26.563114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:26.567972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:26.603875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] Creating event network-vif-plugged:2f27510e-988b-4adf-8776-39c67b3541ce for instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f on hv-cinder-85837 Sep 19 16:17:26.624268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-003bf57f-e75f-4517-b465-f6e5387a37f0 req-323784b9-dffb-44ad-864c-81361974fd27 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.092614 Sep 19 16:17:26.625931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 268/517] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:26.948954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:26.952040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:26.975637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:26.976868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:26.977887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:26.979764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:26.980867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:26.984207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:27.029038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:27.029752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:27.044195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:27.045160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:27.046060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:27.049003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:27.049693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:27.051950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:27.332429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:27.346798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:27.348702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:27.350496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-854f82ed-31e9-4831-9ffa-4317dc6a82ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.326283 Sep 19 16:17:27.351666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 269/518] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:27.392694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:27.404515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4a887a39-7575-4dee-b228-0cb41f26101a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.466782 Sep 19 16:17:27.406035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 250/519] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:28.388438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:28.389380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:28.408645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:28.411861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:28.413612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:28.418811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:28.420458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:28.425279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:28.456539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:28.458538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:28.484204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:28.489756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:28.495791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:28.504060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:28.506408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:28.509515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:28.527753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:28.870107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:28.895274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:28.897043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:28.913041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:28.924489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4a0fc4c0-d744-40f8-9076-80fb117286ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.540293 Sep 19 16:17:28.925107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 270/520] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 542 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:28.926874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e452b55-319d-4fc9-a0e7-41cdd1d0d12d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1318 microversion: 2.1 time: 0.474795 Sep 19 16:17:28.944199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 251/521] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1318 bytes in 493 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:29.953238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:29.955530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:29.968291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:29.969124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:29.972287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:29.974279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:29.975893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:29.979090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:29.981802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:29.983124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:30.038014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:30.040413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:30.043518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:30.047738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:30.052404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:30.058633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:30.220283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:30.234983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-05d3fadd-166b-4ec1-b4c7-70b1dc97a78e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1507 microversion: 2.1 time: 0.285894 Sep 19 16:17:30.246576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 271/522] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1507 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:30.535295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:30.546765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:30.564174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:30.565399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4590febb-9794-4035-95a6-b1187c0d6369 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.594992 Sep 19 16:17:30.565979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 252/523] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 597 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:31.603732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:31.604408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:31.618162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:31.619177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:31.619808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:31.620799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:31.621454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:31.622076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:32.003936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:32.047983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:32.053634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:32.059571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-be9d2e0e-7418-40a7-acd6-02e453bb55b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.467099 Sep 19 16:17:32.060177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 272/524] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 469 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:33.092842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:33.093755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:33.102251 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:33.118569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:33.119896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.017s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:33.121482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:33.130987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:33.132569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:33.450784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:33.469829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:33.471596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:33.472907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-db0301a6-7bd4-4c27-8ff2-00c0983cdc70 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.383968 Sep 19 16:17:33.482406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 253/525] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 391 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:34.508096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:34.517985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:34.539798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:34.540718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:34.541371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:34.551284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:34.552478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:34.556652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:34.852605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:34.878239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:34.880876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:34.882887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6273bebf-2a69-43b1-8b5b-522072bbb6fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.386147 Sep 19 16:17:34.884580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 273/526] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:35.942880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:35.945859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:35.963574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:35.965015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:35.965907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:35.967307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:35.968222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:35.972980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:36.273163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:36.289313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:36.298163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:36.301445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-78b04c46-ad0c-4fcf-b76a-e4ab9bc2d88e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.366640 Sep 19 16:17:36.303249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 254/527] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:37.032337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:37.033321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:37.040077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:37.041019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:37.041708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:37.042690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:37.043361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:37.043994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:37.240915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lazy-loading 'services' on Instance uuid bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:17:37.364097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:37.365518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:37.375699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:37.376207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:37.376655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:37.377408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:37.377868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:37.378307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:37.542845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a9ff8fea-91ed-4513-bbe7-18873269d90f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments/a330688a-4c62-4472-a9e9-6ec347d71d1e" status: 202 len: 0 microversion: 2.1 time: 0.514328 Sep 19 16:17:37.552372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 274/528] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:17:37 2022] DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments/a330688a-4c62-4472-a9e9-6ec347d71d1e => generated 0 bytes in 525 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:17:37.786589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:37.804430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:37.807417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:37.809751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-68e6f7af-d869-4219-8439-1286634b979a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.457942 Sep 19 16:17:37.811218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 255/529] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:38.548240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-49697f2d-342f-4430-ac3b-eb099dd2f2a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'create', calling method: .version_select at 0x7fadbf943e50>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-2051475603", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJJaCST6IXCRmECP9mgttBa/0964K3CFZEFXhn+mFISKkdA+du4OTMIOY6nAu9A1/Bsc9Yp01B01kdTY6DkljEYvtuXLHjwcHpdmvfbJFfAUjfpOOq/xASirCwqpoZTz9A=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:38.591448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-49697f2d-342f-4430-ac3b-eb099dd2f2a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Getting quotas for project 027b06570b2a41c4a0b43f3529340077. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:17:38.597095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-49697f2d-342f-4430-ac3b-eb099dd2f2a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Getting quotas for user 960704dfe18c4d739e90d00288724983 and project 027b06570b2a41c4a0b43f3529340077. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:17:38.630535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-49697f2d-342f-4430-ac3b-eb099dd2f2a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 403 microversion: 2.1 time: 0.090800 Sep 19 16:17:38.631553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 275/530] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:17:38 2022] POST /compute/v2.1/os-keypairs => generated 403 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:38.733686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-1138645370", "imageRef": "177f475e-78ac-4246-be40-c3c56a72e3a7", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-2051475603", "networks": [{"uuid": "fabdb99d-f722-42ae-b412-541c99cb7975"}]}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:38.872747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:38.873928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:38.892038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:38.893415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:38.894976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:38.897077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:38.898298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:38.899389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:38.924762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] validate_networks() for [('fabdb99d-f722-42ae-b412-541c99cb7975', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:17:39.269779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:39.289882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:39.292903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:39.295164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e2f94c0c-5762-47cb-b999-9ec7d03ba6bf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.427117 Sep 19 16:17:39.296722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 276/531] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:39.829771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:17:39.830541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:17:39.831399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:17:39.832184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:17:39.832876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:17:39.851662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:39.852549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:39.853320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:39.854638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:39.855430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:39.856281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:39.902288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Getting quotas for project 027b06570b2a41c4a0b43f3529340077. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:17:39.915132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Getting quotas for user 960704dfe18c4d739e90d00288724983 and project 027b06570b2a41c4a0b43f3529340077. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:17:39.938419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:17:39.941442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='177f475e-78ac-4246-be40-c3c56a72e3a7',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:17:40.033177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b76d3398-0ef3-48b7-b10b-81f6c5002923 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 1.304602 Sep 19 16:17:40.034182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 256/532] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:17:38 2022] POST /compute/v2.1/servers => generated 380 bytes in 1307 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 19 16:17:40.052721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-af3a3608-0281-4a55-8dda-d917e3b0e25e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:40.053790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-af3a3608-0281-4a55-8dda-d917e3b0e25e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:40.166573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-af3a3608-0281-4a55-8dda-d917e3b0e25e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:40.188416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-af3a3608-0281-4a55-8dda-d917e3b0e25e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1259 microversion: 2.1 time: 0.140955 Sep 19 16:17:40.189725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 277/533] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:40 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1259 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:40.347939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:40.366118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:40.366118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:40.366118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:40.366118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:40.366822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:40.366822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:40.366822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:40.654870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:40.698684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:40.701854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:40.704088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1431ceec-e730-4bef-9d46-31b19592f795 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.363741 Sep 19 16:17:40.736515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 257/534] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:41.221740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:41.224301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:41.236208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:41.237121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:41.237738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:41.238721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:41.239669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:41.240205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:41.330094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:41.350108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a189fa6b-db01-48bc-99e6-5cdaf7aec95a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1263 microversion: 2.1 time: 0.137032 Sep 19 16:17:41.351396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 278/535] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:41 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1263 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:41.768048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:41.769162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:41.775030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:41.784402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:41.785183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:41.785980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:41.787318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:41.788112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:41.788883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:41.982862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:41.999905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:41.999905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:41.999905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-67d9a157-3ae6-4044-9127-7ef43a861e96 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.242930 Sep 19 16:17:42.000760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 258/536] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:42.389170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:42.389909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:42.398823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:42.399323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:42.399766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:42.400665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:42.401496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:42.402176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:42.475860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:42.484321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8470d886-06fd-40fc-91f9-0c46aca53105 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1263 microversion: 2.1 time: 0.099777 Sep 19 16:17:42.484932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 279/537] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:42 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1263 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:43.025719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:43.026435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:43.033164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:43.033644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:43.034069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:43.034832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:43.035281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:43.035698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:43.227504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:43.237548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:43.239108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:43.240315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1e361700-b012-45f8-901f-9fb34e8d4f1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.218547 Sep 19 16:17:43.240879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 259/538] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:43.514538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:43.526066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:43.541207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:43.541791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:43.542282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:43.543244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:43.543867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:43.544333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:43.617263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:43.625706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1941354f-bca1-488e-9a72-3d0a6c7410b5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1255 microversion: 2.1 time: 0.119253 Sep 19 16:17:43.626302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 280/539] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:43 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1255 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:44.267713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:44.269547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:44.279232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:44.279981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:44.280576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:44.282065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:44.282811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:44.283843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:44.470463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:44.487491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:44.489326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:44.491872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5b0c6daa-ba26-4c74-ac33-3f8a3a608e40 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.228897 Sep 19 16:17:44.492579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 260/540] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:44.649343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:44.649959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:44.656713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:44.657189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:44.657614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:44.658306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:44.658785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:44.659199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:44.725911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:44.734847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4d35cedd-e8a1-494a-8a80-7760c770b43f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1311 microversion: 2.1 time: 0.088791 Sep 19 16:17:44.735449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 281/541] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:44 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1311 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:45.164820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:45.165790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:45.179325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.180530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.183644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.183644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.184024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.184563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.239116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1b1b0787-68ad-4a34-87fa-bf8dfaba989e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/ips" status: 200 len: 150 microversion: 2.1 time: 0.082999 Sep 19 16:17:45.241182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 261/542] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:17:45 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/ips => generated 150 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:17:45.261617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "652d6b02-71d3-4c9f-a512-1e3c3c0ce750"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:45.264427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:45.276053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.277052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.277892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.279573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.280488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.281386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.463913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: POST /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/os-volume_attachments HTTP/1.1 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 74 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:17:45.467930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {"volumeAttachment": {"volumeId": "652d6b02-71d3-4c9f-a512-1e3c3c0ce750"}} Sep 19 16:17:45.470982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fa7d958b-1b74-4d02-9a97-aae5da88ce3e', 'body': {'volumeAttachment': {'volumeId': '652d6b02-71d3-4c9f-a512-1e3c3c0ce750'}}} Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:17:45.472491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:17:45.475834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:17:45.478337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 400 to user: Invalid volume: volume 652d6b02-71d3-4c9f-a512-1e3c3c0ce750 is already attached to instances: 5d5cfe70-1a06-4c64-91bf-4681779bd260 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:17:45.479208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:45.479907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b4e2679b-167f-4bb5-890c-4e265b8cce6e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.220002 Sep 19 16:17:45.479907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 282/543] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:17:45 2022] POST /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e/os-volume_attachments => generated 174 bytes in 222 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:17:45.481264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.481264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.483513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.488236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:45.490426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:45.501198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.501198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.501925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.503246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.504115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.504910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.527929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Going to try to stop instance {{(pid=120663) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 19 16:17:45.550093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:17:45.627927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b8730169-3161-4755-89f3-cd1984e4ffbd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action" status: 202 len: 0 microversion: 2.1 time: 0.170362 Sep 19 16:17:45.635090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 262/544] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:17:45 2022] POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action => generated 0 bytes in 181 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:17:45.645485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:45.646195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:45.676393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.677747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.678785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.680157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.681144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.682166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.699519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69f29360-a411-4b40-bcf9-e59aca670c49 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 204 len: 0 microversion: 2.1 time: 0.218315 Sep 19 16:17:45.701548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 283/545] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:17:45 2022] DELETE /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:17:45.707838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:45.708581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:45.715740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.716658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.717294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.718226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:45.718943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:45.719544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:45.945520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:45.983961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69d3a990-6c83-4f9c-911f-53027cd5aad0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.280549 Sep 19 16:17:45.987998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 284/546] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:45 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:46.014839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:46.018731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:46.039240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:46.046121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:46.047688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:46.049573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:46.051674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:46.052270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:46.083102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:46.120501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:46.122346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:46.123731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c3752588-9096-4283-b97b-e555572aa43c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.482846 Sep 19 16:17:46.139925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 263/547] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 504 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:46.153995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:46.154903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:46.163047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:46.163534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:46.164181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:46.165080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:46.165699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:46.166285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:46.296857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:46.311250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-34bb656e-a059-4fc8-ad34-b43ca1cbc1a5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.320006 Sep 19 16:17:46.311855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 285/548] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:45 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:46.315988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:46.333420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-56ce381f-ca62-4224-8c96-1b58b024cd37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1317 microversion: 2.1 time: 0.191589 Sep 19 16:17:46.334835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 264/549] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:46 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1317 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:47.028653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:47.029843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:47.033137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=74,nova_api:UPDATE=2,nova_api:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:47.039720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.040223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.040693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.041977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.041977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.042340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.068602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:47.159148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:47.160429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:47.170185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.171350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.172148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.173345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.174167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.175045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.184098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:47.205907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.227609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-08115f2c-49c2-4049-968f-881dbe0c6dee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.203695 Sep 19 16:17:47.229189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 286/550] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:47 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:47.352241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:47.355075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:47.366206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.384853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.385423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.386349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.386679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.388105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.389625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.391487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8d36d05b-dff2-4846-b89e-53d180f8d57c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.239995 Sep 19 16:17:47.403074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 265/551] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:47.403524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.404297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.429107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:47.429745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:47.445805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.446329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.447079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.448230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:47.448888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:47.451765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:47.674064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.687593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-726db6b7-6b75-46df-86d4-d5ddba74a421 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.343458 Sep 19 16:17:47.688539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 287/552] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:47 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:47.883182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:47.898326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9d35de9f-2580-461e-a24b-d1fe552116d4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.493604 Sep 19 16:17:47.899605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 266/553] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:47 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:48.254741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:48.256128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:48.267742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.268878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.270039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.271741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.272839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.273812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.427223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:48.428906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:48.443364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.444777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.446247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.447567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.447689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:48.448398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.449446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.457899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f176218-db9d-47ce-9657-a725e3df2252 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.209042 Sep 19 16:17:48.459391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 288/554] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:48 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:48.649742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:48.662667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:48.664195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:48.665394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-918cdec1-7247-4f95-a67e-6876fce58660 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.247602 Sep 19 16:17:48.665958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 267/555] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:48.717474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:48.718718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:48.738408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.739325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.740110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.741412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.742731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.743758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.942239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:48.943456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:48.949426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:48.954942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.955746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.956457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.957676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:48.958435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:48.959172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:48.963375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ba6637c4-528c-4a03-b4c9-0a76eb278c63 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.253599 Sep 19 16:17:48.964710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 289/556] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:48 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:49.139899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:49.158022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b2e4ecc8-0eae-4ea3-a257-5ab5023a0d3e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.223032 Sep 19 16:17:49.159574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 268/557] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:48 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:49.507453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:49.509157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:49.517138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.518012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.518731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:49.519641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.520293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.520806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:49.699714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:49.700826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:49.714605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.716008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.716714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:49.717710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.718506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.718953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:49.762680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:49.773692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e672005-2239-473a-a63f-af8183380d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.273197 Sep 19 16:17:49.775569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 290/558] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:49 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 276 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:49.873665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:49.927865 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:49.987578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:49.988211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:49.995544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.996037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.996607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:49.997459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:49.998269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:49.998741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:50.004458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:50.020264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:50.023227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:50.025514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3dce0cb6-941c-46bb-a767-60c2260e197c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.334913 Sep 19 16:17:50.026773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 269/559] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:50.161836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:50.176381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c2cd9bf-ea70-41b6-b28a-e39d52c3c0aa tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.192406 Sep 19 16:17:50.177241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 291/560] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:49 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:50.200477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:50.201591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:50.212803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:50.213657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:50.214682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:50.216047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:50.216901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:50.217802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:50.468539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:50.481610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c16168a5-5f5d-48f4-967d-908677351f82 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.287874 Sep 19 16:17:50.482927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 270/561] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:50 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:50.820002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:50.821813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:50.824638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=6 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:50.835045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:50.836201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:50.837260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:50.847154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:50.848115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:50.848726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.077106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:51.087033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:51.112755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.114662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.116581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.118862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.120490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.122413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.154713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.172385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1ce7ae38-b1c8-44cf-a811-46b2bd6d4b35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.360992 Sep 19 16:17:51.190010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 292/562] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:50 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:51.212212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:51.213337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:51.222227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.222849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.223586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.225053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.225945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.226767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.467803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.479262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.480597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.481816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ea5160d0-88af-4bda-9e89-2ba494e04f60 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.413013 Sep 19 16:17:51.482463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 271/563] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:51.518259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.523765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:51.524845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:51.537393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d1c67661-ea5e-4221-a1be-f870b6b86181 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.331602 Sep 19 16:17:51.537825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.538944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 293/564] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:51 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:51.539159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.539540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.541108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:51.541970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:51.542800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:51.864550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:51.880880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-189e93b3-6bfa-4e6d-9166-f51575f035e6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.363662 Sep 19 16:17:51.890927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 272/565] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:51 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:52.210252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:52.211497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:52.228030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.228785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.229457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.230631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.231344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.231950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.488841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:52.499715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ff89aded-8ea3-400c-a38c-12ae54155543 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.295660 Sep 19 16:17:52.500453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 294/566] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:52 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 298 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:52.527364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:52.529155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:52.537957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.538714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.539696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.540566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.541429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.542469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.586309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:52.587488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:52.598706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.599599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.600402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.601696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.602668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.603386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.826886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:52.850230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:52.854862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:52.859827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cd5ca77c-66e5-4795-840e-304f566c37fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.337650 Sep 19 16:17:52.862806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 273/567] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:52.874468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:52.887062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7bf205e8-289e-4e08-93cc-1698d03de056 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.309022 Sep 19 16:17:52.887655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 295/568] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:52 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:52.922233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:52.923658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:52.935405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.936267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.937610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:52.938935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:52.939871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:52.940648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.197193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:53.210780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-db5543b4-ae72-46ac-bab3-3f35dd1a2225 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.295458 Sep 19 16:17:53.216140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 274/569] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:52 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:53.543853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:53.544982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:53.563252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.564137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.565160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.566514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.567354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.568154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.796128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:53.811175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-383baa17-dfea-4d0e-ac4c-42eca975cec2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.274283 Sep 19 16:17:53.812960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 296/570] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:53 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:53.889044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:53.890620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:53.898820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.899309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.899802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.900550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.901002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.901521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.920539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:53.922345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:53.933285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.934304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.935028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:53.936298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:53.937633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:53.938410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:54.215864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:54.232003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:54.246538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:54.246538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3563f0b2-36ea-46d4-beb3-a1c31c93ad08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.350534 Sep 19 16:17:54.246538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 275/571] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 352 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:54.246538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:54.247316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:54.262681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:54.263177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:54.263619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:54.264641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:54.265139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:54.265586 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:54.293476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:54.327293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0cc600e3-d57c-4a74-80a6-dbd68241be64 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.408673 Sep 19 16:17:54.329495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 297/572] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:53 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:54.561822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:54.572518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c645d517-e19f-40fe-805a-89f4f77cd7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.334704 Sep 19 16:17:54.573622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 276/573] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:54 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:54.837984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:54.838952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:54.848040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:54.848834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:54.849550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:54.851178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:54.851929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:54.852859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.072672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:55.085234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c8aabcfe-7b2c-4b30-8c6d-dfe02b78ded6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.251569 Sep 19 16:17:55.085959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 298/574] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:54 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:55.292008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:55.293324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:55.306955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.307831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.308657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.309983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.310884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.311668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.384929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:55.388623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:55.413256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.414093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.414968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.416199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.417303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.417819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.560903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:55.584509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:55.592052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:55.592052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-855c6857-e8a7-4c22-9a5a-2f715924dd1e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.306258 Sep 19 16:17:55.592474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 277/575] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:17:55.602689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "d94f3e7c-f979-4419-884f-8141e3deebc9"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:55.607112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:55.619309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.620161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.620921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.626443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.626443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.626443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.937664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:55.947128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-492f1ac7-1474-4f07-b3af-a50e8ddfe698 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.571988 Sep 19 16:17:55.947756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 299/576] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:55 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 574 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:55.953433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:55.956712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:55.972717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.973213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.973648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:55.974401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:55.974846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:55.975406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:56.183430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:56.194807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9a1dc628-93e3-4fd2-b2e2-eb9a140715c7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.245704 Sep 19 16:17:56.195623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 300/577] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:55 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:56.201297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:56.202173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:56.217020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:56.218778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:56.221357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:56.223326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:56.224796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:56.225369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:56.410631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:56.433877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a0370ae-3971-49d8-8e0c-d3f9106d9e6a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.236678 Sep 19 16:17:56.435013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 301/578] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:56 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:56.654662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:56.669914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:56.697714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:56.698578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:56.699354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:56.700935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:56.701588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:56.702185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:56.916178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:56.930655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:56.932505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:56.933742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a97cea0d-7acd-40c2-ae8a-b7419b3e0a62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.300528 Sep 19 16:17:56.934720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 302/579] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:56.991279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:56.992250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:56.999490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.000159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.000730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.001658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.002234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.002912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.011064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:17:57.199771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:57.217067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-510e0364-2238-4a1c-b6c7-76f3d848271d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.230104 Sep 19 16:17:57.218089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 303/580] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:56 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:57.225581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:57.226306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:57.237408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.237887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.238325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.239106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.239566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.239998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.423942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:57.433401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-05e575c0-3a56-451c-ab96-360f5f7f37af tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.213723 Sep 19 16:17:57.434001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 304/581] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:57 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:57.463517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:57.464586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:57.475768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.476628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.477410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.478698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.479442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.480150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.630757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:57.639536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c78e0fd7-a394-4f5e-afec-fd7aee3befc5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.183073 Sep 19 16:17:57.640739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 305/582] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:57 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:57.982768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:57.983676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:57.995430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.996330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:57.997228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:57.998627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:57.999509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.000810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.202096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:58.212833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:58.214654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:58.215979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e479199e-0d0f-435b-914a-80d3c9381b46 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.238593 Sep 19 16:17:58.216571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 306/583] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:17:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:17:58.243795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:58.244709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:58.251841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.252487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.252926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.255051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.255529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.255963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.456978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:58.481758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2821fc08-cf1e-4281-9e0a-8fe94520d389 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1683 microversion: 2.1 time: 0.242198 Sep 19 16:17:58.482873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 307/584] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:58 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1683 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:17:58.506779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:58.508023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:58.541096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.541374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.541939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.542863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.543441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.543970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.796742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:17:58.806179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38cd86b2-9060-4c12-aeeb-da5444a2295a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.314024 Sep 19 16:17:58.807179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 308/585] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:58 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:17:58.816693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "3ec19191-d0b9-429b-b5bc-18ac7f02af93", "device": "/dev/sdb"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:17:58.822360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:58.834972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.836281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.837085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:58.838089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:58.838974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:58.839612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:59.793992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-22e7c26c-72c4-4972-8f20-49053536e998 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.198949 Sep 19 16:17:59.795127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 278/586] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:17:55 2022] POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 194 bytes in 4202 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:59.817114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:17:59.821494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:17:59.842070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:59.846883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:59.847560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:59.863594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:17:59.865469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:17:59.866615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:17:59.959574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e456469-80c0-4a1b-938b-905475a83bcb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "POST /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.150281 Sep 19 16:17:59.960583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 309/587] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:17:58 2022] POST /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments => generated 194 bytes in 1153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:17:59.986168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:00.033646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.034537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.035276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.078734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] Creating event network-changed:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:18:00.105375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-d51c2713-6b24-4f90-9fa4-07d1b44e68f6 req-211634f3-bf0c-4044-9590-672a14e9bdf3 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.138571 Sep 19 16:18:00.106541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 310/588] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:17:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:00.117553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:00.118864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:00.132960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.133838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.134675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.135937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.136761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.137507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.437935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:00.479015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:00.482526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:00.485712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-50a5bf89-9f2f-4c61-b7d2-9f5d2bb1735b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.376740 Sep 19 16:18:00.486991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 311/589] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:00.505570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:00.508798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:00.527820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.528923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.531565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.533393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.535372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.538805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.721428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:00.732848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4344c012-cee4-4377-9c90-6011b5c5dea6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1615 microversion: 2.1 time: 0.242929 Sep 19 16:18:00.739055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 312/590] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:00 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1615 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:00.746945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:00.748547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:00.764054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.766014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.767063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.769636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:00.769972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:00.771135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:00.772146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:00.779961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b6beab78-fafc-412f-9e03-69c7a1f8058d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.982491 Sep 19 16:18:00.787188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 279/591] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:17:59 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 991 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:01.063782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:01.096389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7999e7ea-1601-4d25-9a55-326456122481 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.350463 Sep 19 16:18:01.110406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 313/592] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:00 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:01.278858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fa7d958b-1b74-4d02-9a97-aae5da88ce3e", "name": "network-vif-deleted", "tag": "4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:01.289876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.290357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.290827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.305189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] Creating event network-vif-deleted:4ef6e28d-2b2a-4f27-8d15-94a3a2e2af7e for instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e on hv-cinder-85837 Sep 19 16:18:01.314021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-e552725f-a59c-45f0-9b24-c64fc1b558d2 req-e9a3a411-6d77-4206-9f55-6ab093015fd2 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038767 Sep 19 16:18:01.315061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 280/593] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:18:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:01.517988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:01.518810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:01.525714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.526204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.526667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.527363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.527810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.528229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.697224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:01.707699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:01.709226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:01.710458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4125e012-86dc-4bb3-956a-85f64f2047cd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.196937 Sep 19 16:18:01.719100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 314/594] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:01.790129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:01.790979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:01.798852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.799352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.800011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.800775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.801237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.801697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.807074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:01.830412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:01.831469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:01.842636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.844448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.845413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.846847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:01.847856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:01.848872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:01.887903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:01.899432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7a207f8e-9b3b-43f6-b96a-db224ba0614d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1615 microversion: 2.1 time: 0.113959 Sep 19 16:18:01.901167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 281/595] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:01 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1615 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:02.064652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:02.079189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-990ad94f-e842-4fe2-8552-cce21723fac1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.254279 Sep 19 16:18:02.080312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 315/596] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:01 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:02.142410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:02.143555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:02.152660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.153645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.154176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:02.155976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.156565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.157095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:02.571259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:02.584295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-250d9690-2dc4-4478-8e27-af142996f9f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.448922 Sep 19 16:18:02.585749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 282/597] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:02 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:02.744397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:02.747489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:02.789926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.790995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.791555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:02.793183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.800670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.801412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:02.936210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:02.936883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:02.949350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.949848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.950291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:02.951042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:02.951561 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:02.951992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.032602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.041174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.041809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.042298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lazy-loading 'fault' on Instance uuid fa7d958b-1b74-4d02-9a97-aae5da88ce3e {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:18:03.176810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.178650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.188623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.190324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.190593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c0a3c2ef-169f-4450-bd8d-ac7b9353e585 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 200 len: 1332 microversion: 2.1 time: 0.258609 Sep 19 16:18:03.191130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 283/598] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:02 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 1332 bytes in 260 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:03.198700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "name": "network-vif-plugged", "status": "completed", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:03.204965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cabfc1d7-0ae2-4d62-9d63-8a08906d8231 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.471457 Sep 19 16:18:03.214628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 316/599] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:03.227476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.230131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.230794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.237156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:03.238733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:03.250050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.251044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.251754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.252614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.254152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.254774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.259240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] Creating event network-vif-plugged:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:18:03.265511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-e28415ce-5d25-48c4-9cd9-1df5e3dec1f8 req-9acdff70-ea8b-4225-8ce3-5a2bbc27767b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.072409 Sep 19 16:18:03.267646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 284/600] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:18:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:03.553704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.576827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-51935d64-84fa-4d83-bf2e-6b3177b39faa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1677 microversion: 2.1 time: 0.360232 Sep 19 16:18:03.578690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 317/601] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:03 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1677 bytes in 363 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:03.642636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:03.645294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:03.667311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.668697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.672255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.677710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:03.678537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:03.684200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:03.975448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:03.986075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fd31a1d3-13e0-4fda-a533-b930c6e3abeb tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.354705 Sep 19 16:18:03.986737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 285/602] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:03 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:04.241215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:04.242563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:04.244611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:04.245715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:04.251798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.252289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.252724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.253667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.254133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.254594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.267842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.269005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.270001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.271749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.272788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.273741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.281722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e HTTP/1.1 Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:18:04.283691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:18:04.286520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'fa7d958b-1b74-4d02-9a97-aae5da88ce3e'} Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:18:04.288158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:18:04.290883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:04.294770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:18:04.294770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ceed7482-fcf6-4a00-ba87-3e287d3e008a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 404 len: 111 microversion: 2.1 time: 0.049451 Sep 19 16:18:04.294770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 318/603] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:04 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:18:04.301020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ce7a023f-3e54-4dcf-9b51-58f908e9413e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '.version_select at 0x7fadbf867700>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:04.338841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ce7a023f-3e54-4dcf-9b51-58f908e9413e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1521587055" status: 202 len: 0 microversion: 2.1 time: 0.045317 Sep 19 16:18:04.339868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 319/604] 10.222.0.57 () {58 vars in 1221 bytes} [Mon Sep 19 16:18:04 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1521587055 => generated 0 bytes in 47 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:18:04.536857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:04.555000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:04.558518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:04.561346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-19fe1297-614f-40cb-a182-78b0ed805b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.323277 Sep 19 16:18:04.563736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 286/605] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 327 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:18:04.616496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:04.618045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:04.627546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.628561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.629201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.630282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:04.631277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:04.633191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:04.961657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:04.965445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:04.973132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c8df8d58-6f67-4160-97d7-3a7017b579eb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1654 microversion: 2.1 time: 0.369129 Sep 19 16:18:04.973729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 320/606] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:04 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1654 bytes in 370 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:05.004575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:05.013914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:05.015446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:05.030148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.030719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.031164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.033903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.034462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.035283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.036200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.036822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.037404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.082999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Going to try to start instance {{(pid=120663) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 19 16:18:05.151776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4f1fa7a5-d95e-40e8-803f-53bf1491f8fa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action" status: 202 len: 0 microversion: 2.1 time: 0.155225 Sep 19 16:18:05.153659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 287/607] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:18:04 2022] POST /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/action => generated 0 bytes in 159 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:18:05.166609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:05.168251 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:05.180135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.181345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.182363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.183942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.185197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.186182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.295104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:05.304252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dd040633-a5cb-4070-a3c4-ce8c01075403 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.294400 Sep 19 16:18:05.304841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 321/608] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:05 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:05.309866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "name": "network-vif-plugged", "status": "completed", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:05.321135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.321711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.322138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.344643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] Creating event network-vif-plugged:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:18:05.350440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-8a92d410-7607-426c-b52a-76b1e0743c59 req-be96e964-dde4-45fb-ba87-f19853213356 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.044350 Sep 19 16:18:05.351050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 322/609] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:18:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:05.522257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:05.538923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-125a08d6-2a3f-4d59-a0a2-682ac4367435 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1663 microversion: 2.1 time: 0.379958 Sep 19 16:18:05.539967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 288/610] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:05 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1663 bytes in 383 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:05.614873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:05.615830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:05.629001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.629963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.630787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.632245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.633492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.634545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.897123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:05.911953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:05.914006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:05.917702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f39cac09-0298-4278-9d36-119c365b4de4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.308846 Sep 19 16:18:05.918410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 323/611] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:05.962127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:05.962714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:05.970003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.970721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.971285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:05.972099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:05.972657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:05.973184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.125692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lazy-loading 'services' on Instance uuid 5d5cfe70-1a06-4c64-91bf-4681779bd260 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:18:06.276779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-25d6c10e-1143-43b1-a60b-cbf4fff722c4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments/652d6b02-71d3-4c9f-a512-1e3c3c0ce750" status: 202 len: 0 microversion: 2.1 time: 0.318786 Sep 19 16:18:06.277391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 289/612] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:18:05 2022] DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments/652d6b02-71d3-4c9f-a512-1e3c3c0ce750 => generated 0 bytes in 320 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:18:06.286922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:06.289246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:06.296460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.297162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.297374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.298194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.298796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.299086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.332633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:06.333313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-81cba502-443b-4ac4-abf8-52a382998b70 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050475 Sep 19 16:18:06.333781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:06.339439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 324/613] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:06 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:06.345582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.346651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.347345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.348483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.349204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.349888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.581357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:06.584277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:06.597037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.597954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.598795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.600057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.600871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.601683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.668950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:06.689644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2aa4212a-2821-45ff-b4f7-be91f2cbfb7d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.364715 Sep 19 16:18:06.690333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 290/614] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:06 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 367 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:06.859780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:06.875841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-27a74dc3-2a8a-4324-ab70-015fb0ace8da tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1663 microversion: 2.1 time: 0.302984 Sep 19 16:18:06.877358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 325/615] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:06 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1663 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:06.972388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:06.974463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:06.991669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.992841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.993799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:06.995597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:06.996507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:06.997412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.201097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:07.211348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:07.212798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:07.214001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-296efd94-d7c5-4740-adc9-3d55d0302597 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1810 microversion: 2.20 time: 0.256641 Sep 19 16:18:07.214599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 291/616] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1810 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:07.348048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:07.350888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:07.358913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.359400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.359833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.360538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.360991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.361413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.397044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ea2cf5b7-8178-461c-96f7-f460e5ecfb6b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053265 Sep 19 16:18:07.402364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 326/617] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:07 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:07.744741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:07.747228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:07.777296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.781775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.782797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.784970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.785601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.786334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.925784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:07.926971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:07.938826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.940117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.940917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.942183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:07.943116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:07.944025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:07.992426 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:08.005592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e8bcc2fb-ff08-49e3-8848-d510c64e36b4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.277443 Sep 19 16:18:08.006528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 292/618] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:07 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:08.127712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:08.140931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f6e5efd-6897-4e55-9b2b-23b79e9fda89 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1663 microversion: 2.1 time: 0.222690 Sep 19 16:18:08.142065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 327/619] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:07 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1663 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:08.244680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:08.245946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:08.258124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.259172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.260022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.261392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.262323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.263220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.413850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:08.417884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:08.430701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.431276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.432097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.433297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.434120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.435002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.458428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:08.469817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:08.471216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:08.472385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-de326464-02e4-4148-9adc-e8917ed87e7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1797 microversion: 2.20 time: 0.235332 Sep 19 16:18:08.472942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 293/620] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:18:08 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1797 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:18:08.475342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-babe3dad-045f-485c-9cc9-02dea1ff23ca tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068862 Sep 19 16:18:08.476178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 328/621] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:08 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:08.907823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:08.912945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:08.920646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.921509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.922016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.922823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:08.923274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:08.923695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:08.967531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'flavor' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:18:09.047371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:09.048475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:09.057295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.058129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.058944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.060853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.061778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.062644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.121375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:09.154819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c001c5c0-e70d-4e5f-88b5-7dba2effd48f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action" status: 200 len: 32 microversion: 2.1 time: 0.252491 Sep 19 16:18:09.156614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 294/622] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:18:08 2022] POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action => generated 32 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 19 16:18:09.176073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:09.177099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:09.184037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.184682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.185224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.186075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.187154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.187659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.256987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:09.265809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-619c00f6-7652-4aae-82d9-de8a4cfb30dd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1359 microversion: 2.1 time: 0.224702 Sep 19 16:18:09.266417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 329/623] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:09 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1359 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:09.271384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:09.272001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:09.278804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.281112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.281571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.282315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.282787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.283393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.388848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:09.405929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-560c2975-2994-4772-95a9-867fc2b1563c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1663 microversion: 2.1 time: 0.238175 Sep 19 16:18:09.406995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 295/624] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:09 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1663 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:09.482088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:09.490028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:09.493903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:09.501642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-24c03209-8eb1-4cf1-919a-21343d0bbd6b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.233330 Sep 19 16:18:09.503808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 330/625] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:09 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:09.505905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.506511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.507104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.507930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:09.508485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:09.509169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:09.549370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ecff5b56-2ef9-437f-b24d-034b110ac2e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064341 Sep 19 16:18:09.550118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 296/626] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:09 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:10.300295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.301000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:10.308009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.308493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.308924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.309642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.310089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.310542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.446892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.447572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:10.454546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.455053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.455491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.456514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.456983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.457413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.475140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:10.485169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7061e765-c003-439d-a0f2-cd1ff4493d15 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1549 microversion: 2.1 time: 0.189672 Sep 19 16:18:10.486189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 331/627] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1549 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:10.508776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.509491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:10.517051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.517831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.518262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.519030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.519503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.519905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.649868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:10.665702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8c1d9cd0-7232-476f-a665-ef33f8130d64 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1663 microversion: 2.1 time: 0.222316 Sep 19 16:18:10.666416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 297/628] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1663 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:10.677852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.679175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:10.686112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.686760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.687272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.688069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.688606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.689111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.730670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:10.743586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8da6222e-60d4-4346-a1bc-9082faf819c9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1549 microversion: 2.1 time: 0.238306 Sep 19 16:18:10.746051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 332/629] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1549 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:10.755440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.759830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:10.772367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.773623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.774645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.776145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.777164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.778117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.839705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-44e01677-317f-4661-a420-86d868964a9a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.090667 Sep 19 16:18:10.840740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 333/630] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:10.850410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:10.856817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:10.858031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Searching by: {'deleted': False, 'project_id': '027b06570b2a41c4a0b43f3529340077'} {{(pid=120664) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 19 16:18:10.866849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.867410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.867854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.868604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:10.869060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:10.869486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:10.954003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:10.975815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.multi_cell_list [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120664) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 19 16:18:10.976466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-300a2e4e-f9aa-4377-83e6-62eddac1634c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.305871 Sep 19 16:18:10.977740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 298/631] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 311 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:11.083491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8f6f75b5-7eba-4fb7-b4e9-21052f587bde tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.240947 Sep 19 16:18:11.084098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 334/632] 10.222.0.57 () {58 vars in 1125 bytes} [Mon Sep 19 16:18:10 2022] GET /compute/v2.1/servers => generated 343 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:11.097353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:11.098017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:11.105038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.105573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.106054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.106907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.107428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.107921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.281724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:11.289937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7f5f93ba-3e3c-4625-88f6-18c5b5897eae tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1549 microversion: 2.1 time: 0.197024 Sep 19 16:18:11.290557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 299/633] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:11 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1549 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:11.697494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:11.698242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:11.701631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:11.723722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.724325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.724870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.725654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.726335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.726979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.857667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:11.861550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:11.867728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=86 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:11.875437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.876306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.877080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.878130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:11.879001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:11.879793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:11.908659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:11.922635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-00b97db7-4c6b-449d-b90d-562a15d1ceb5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1667 microversion: 2.1 time: 0.230756 Sep 19 16:18:11.923718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 335/634] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:11 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1667 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:11.932873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ab7b9f9c-e5f1-4013-8acd-982634d53b1a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.082927 Sep 19 16:18:11.933483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 300/635] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:11 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:12.012213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:12.012828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:12.019414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:12.019906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:12.020341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:12.021054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:12.021508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:12.021927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:12.187721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:12.197374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bcdacf11-53e7-40be-9275-a0a0c3c22561 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.189221 Sep 19 16:18:12.197988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 336/636] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:12.945757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:12.948020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:12.957130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:12.958226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:12.958941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:12.959817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:12.960450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:12.961357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:12.996868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ce758d28-b54b-406b-a278-9d29bb453d21 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054963 Sep 19 16:18:12.997825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 301/637] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:12 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:13.019742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "741cfdcd-bd82-4d3e-8f21-5f862f2381f9"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:13.022257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:13.029306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:13.029797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:13.030907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:13.031655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:13.032121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:13.032558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:13.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:13.241040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:13.242151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:13.250786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:13.251353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:13.251945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:13.253104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:13.253656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:13.254181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:13.345592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-603d5718-3532-450e-81a4-29bc80a8cd78 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.329700 Sep 19 16:18:13.346750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 337/638] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:18:13 2022] POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 194 bytes in 332 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:13.461491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:13.475515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9597a4fe-7e98-4866-acc8-9772c5294e49 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.240929 Sep 19 16:18:13.476537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 302/639] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:13 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:14.012658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:14.015433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:14.023429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:14.028834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:14.028834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:14.028834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:14.028834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:14.029444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:14.060983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fb8884ae-90d1-4ba1-816b-8bbd17cc85bf tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053589 Sep 19 16:18:14.062061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 338/640] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:14 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:18:14.519706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:14.519706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:14.531843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:14.532598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:14.533515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:14.535156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:14.536016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:14.537158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:14.703661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:14.712757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5136cc03-4974-4d9d-8e11-8bb5f61e91e1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.204739 Sep 19 16:18:14.713354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 303/641] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:14 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:15.080094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:15.082938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:15.091071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:15.091560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:15.092031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:15.093487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:15.094161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:15.095225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:15.132747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-22c0fd71-edf3-4371-987a-6c084fe7cc02 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060311 Sep 19 16:18:15.133355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 339/642] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:15 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:15.752014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:15.752839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:15.760225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:15.760624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:15.761048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:15.761871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:15.762314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:15.762784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:15.924891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:15.935640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-becfd255-4a2f-4007-8792-2d2ab0d67e4d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.188575 Sep 19 16:18:15.936932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 304/643] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:16.157389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:16.161658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:16.173865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.174794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.175589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.180639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.183463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.184532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.223419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e5c20862-4e9f-4b52-a521-369f7212ccdd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.079864 Sep 19 16:18:16.224006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 340/644] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:16 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260/os-volume_attachments => generated 25 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 19 16:18:16.308999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:16.309540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:16.316656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.317367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.318046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.319334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.320027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.320506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.350362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:18:16.443099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0cfd78a5-f285-4c60-a351-f3a87e115016 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 204 len: 0 microversion: 2.1 time: 0.137414 Sep 19 16:18:16.443727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 305/645] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:18:16 2022] DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:18:16.452930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:16.453446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:16.459978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.460435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.460862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.461582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.462027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.462476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.648654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:16.657357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eaeb2668-2a55-451d-81d7-8699b8c043fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1681 microversion: 2.1 time: 0.209015 Sep 19 16:18:16.658047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 341/646] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:16 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1681 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:16.965148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:16.965924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:16.974156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.974691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.975159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:16.975850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:16.976295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:16.976738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:17.138079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:17.147796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4c669ca4-0a44-4fe4-8d89-669245f90b95 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.186969 Sep 19 16:18:17.148385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 306/647] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:17.682530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:17.683213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:17.690061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:17.690600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:17.691045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:17.691760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:17.692204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:17.692621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:17.878289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:17.890729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-331afc8d-e2fe-435f-aa3c-6efe931ab81c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1681 microversion: 2.1 time: 0.210934 Sep 19 16:18:17.891644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 342/648] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:17 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1681 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:18.172375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:18.173027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:18.181332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:18.181959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:18.182552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:18.183420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:18.183989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:18.184424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:18.345411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:18.354553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ce471dd8-f789-4857-b8c9-8670dd8f3005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.186839 Sep 19 16:18:18.355061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 307/649] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:18 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:18.920516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:18.923330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:18.939826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:18.941059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:18.942098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:18.943481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:18.944337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:18.945167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:19.268509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:19.278024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-90717552-a48e-4601-b341-bf97f12836f7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1681 microversion: 2.1 time: 0.365834 Sep 19 16:18:19.278850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 343/650] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:18 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1681 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:19.378475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:19.379030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:19.386203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:19.386735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:19.387171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:19.387887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:19.388412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:19.388762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:19.518100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "5d5cfe70-1a06-4c64-91bf-4681779bd260", "name": "network-vif-deleted", "tag": "bf337196-4336-4bb9-9fcf-4dfbcb7910a6"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:19.535303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:19.535901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:19.536347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:19.552984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] Creating event network-vif-deleted:bf337196-4336-4bb9-9fcf-4dfbcb7910a6 for instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 on hv-cinder-85837 Sep 19 16:18:19.564213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-fe5bfdb0-fb5c-4fd8-b7ef-79a23e213372 req-ec222dd5-b5a2-49c9-8366-6af261e4655b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051590 Sep 19 16:18:19.569029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 344/651] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:18:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:19.626574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:19.641944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6d104e14-311a-41d0-9539-91c83cb16bfe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.267760 Sep 19 16:18:19.643669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 308/652] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:20.309675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:20.311081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:20.321998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:20.322835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:20.323565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:20.324781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:20.325557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:20.326243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:20.384952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:20.385794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:20.386576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:20.387387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lazy-loading 'fault' on Instance uuid 5d5cfe70-1a06-4c64-91bf-4681779bd260 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:18:20.454273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:20.464503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c4419d48-0e6e-472b-b794-1993de6172de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 200 len: 1332 microversion: 2.1 time: 0.162533 Sep 19 16:18:20.465401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 345/653] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:20 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 1332 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:20.672137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:20.673301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:20.684906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:20.685629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:20.686964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:20.688260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:20.689048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:20.689780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:20.718351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:20.896534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:20.908436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-213fab4c-c7fe-4fd4-87bb-612aecb1d1eb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.243705 Sep 19 16:18:20.909337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 309/654] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:21.505613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:21.506405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:21.508675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:21.515200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:21.515681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:21.516391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:21.517231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:21.517681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:21.518147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:21.533031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 HTTP/1.1 Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:18:21.534756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:18:21.536096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '5d5cfe70-1a06-4c64-91bf-4681779bd260'} Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:18:21.536820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:18:21.539236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:18:21.540886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:18:21.540886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-05d66af7-959d-4b99-8896-239ea9db51ac tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 404 len: 111 microversion: 2.1 time: 0.035864 Sep 19 16:18:21.540886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 346/655] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:21 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:18:21.548753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f048e8ea-f2e3-48ed-8946-af05ae1fa747 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '.version_select at 0x7fadbf920a60>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:21.576035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f048e8ea-f2e3-48ed-8946-af05ae1fa747 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2115728446" status: 202 len: 0 microversion: 2.1 time: 0.032051 Sep 19 16:18:21.576628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 310/656] 10.222.0.57 () {58 vars in 1221 bytes} [Mon Sep 19 16:18:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2115728446 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:18:21.937737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:21.938891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:21.949678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:21.950523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:21.951346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:21.953072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:21.953850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:21.954532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:22.193597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:22.214837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-af4aba84-45f7-4d02-bcc5-b72f181f351c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.283745 Sep 19 16:18:22.216199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 347/657] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:21 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:22.429412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-66ae8865-7bff-4d7a-9131-5bdc68ccff46 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: .version_select at 0x7fadbf902280>, body: {"keypair": {"name": "tempest-keypair-2097141825", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNm2v4J4O/PnNmfVy+OZ1JayPARYq/CdsAKsyUJ24joyBST68jfUaxbu+2Xr9txkVG4rTme7QQRD8smQNtXtVzJ4aqCvBjRg8mxm0kks+aj2a0E9//t4GeQWOGoJlLl0KQ=="}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:22.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-66ae8865-7bff-4d7a-9131-5bdc68ccff46 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:18:22.451662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-66ae8865-7bff-4d7a-9131-5bdc68ccff46 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:18:22.477912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-66ae8865-7bff-4d7a-9131-5bdc68ccff46 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.053045 Sep 19 16:18:22.478549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 311/658] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:18:22 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:18:23.246231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:23.247818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:23.257852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:23.258628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:23.259419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:23.261078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:23.261777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:23.262499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:23.493144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:23.506130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0f20be38-d312-4fab-82d3-b09b6ae87fcd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.267485 Sep 19 16:18:23.506953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 348/659] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:23 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:24.161291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-329496580", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "145a1b1d-9f41-428a-98d1-15b34e55a4ad"}], "security_groups": [{"name": "tempest-securitygroup--1498328001"}], "key_name": "tempest-keypair-2097141825", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:18:24.273061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] validate_networks() for [('145a1b1d-9f41-428a-98d1-15b34e55a4ad', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:18:24.533308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:24.534156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:24.542787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:24.543459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:24.543984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:24.544808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:24.545551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:24.546439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:24.701141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:24.712323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9f9dca13-76e7-4ee0-a076-097500608427 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.184531 Sep 19 16:18:24.713183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 349/660] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:24 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:24.800861 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:18:24.801276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:18:24.801748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:18:24.802671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:18:24.803109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:18:24.813479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:24.814020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:24.814612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:24.815868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:24.816455 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:24.816928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:24.835870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:18:24.840271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:18:24.852243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:18:24.854606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:18:24.907951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2e3feab0-4717-4249-9c8c-de429238bac9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.750778 Sep 19 16:18:24.909073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 312/661] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:18:24 2022] POST /compute/v2.1/servers => generated 402 bytes in 753 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 19 16:18:24.928565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ef3fa5eb-efec-4f27-9867-f5979b05b7db tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:24.929689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ef3fa5eb-efec-4f27-9867-f5979b05b7db tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:24.999108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ef3fa5eb-efec-4f27-9867-f5979b05b7db tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:25.007126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ef3fa5eb-efec-4f27-9867-f5979b05b7db tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1241 microversion: 2.1 time: 0.085938 Sep 19 16:18:25.007861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 350/662] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:24 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1241 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:25.751565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:25.752204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:25.759882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:25.760503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:25.761005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:25.761804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:25.762552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:25.763128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:25.915081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:25.924975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b4ec5664-e0d8-45b3-8f5c-6a13d2e6604c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.178073 Sep 19 16:18:25.925560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 313/663] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:25 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:26.033913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:26.034624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:26.041649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:26.042136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:26.042616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:26.043341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:26.043793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:26.044217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:26.112554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:26.121258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-db081289-ab74-43e1-9e47-33a0fe3429d9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.091346 Sep 19 16:18:26.121921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 351/664] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:26 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:26.948559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:26.949191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:26.957061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:26.957626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:26.958192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:26.959239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:26.959893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:26.960525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:27.145751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:27.148697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:27.149909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:27.160110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3fcb65de-2f16-45d4-8500-c35f05cdaae0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.215229 Sep 19 16:18:27.160588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:27.161105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 314/665] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:26 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:27.161381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:27.162477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:27.163750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:27.164588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:27.165342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:27.254114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:27.262358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0c5672a6-7264-49b1-8892-c73a794f3162 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.120519 Sep 19 16:18:27.262986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 352/666] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:27 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:28.199352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:28.200008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:28.207746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:28.208282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:28.208757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:28.209547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:28.210145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:28.210729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:28.287259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:28.288259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:28.297007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:28.298094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:28.298953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:28.300310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:28.301195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:28.302024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:28.368285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:28.384446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aac307b8-50e6-4b29-9442-1746da7b691b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.188711 Sep 19 16:18:28.385469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 315/667] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:28 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:28.393122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:28.401268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a98c3f48-c922-48ba-aa06-aae1f702f8b9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.119103 Sep 19 16:18:28.401857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 353/668] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:28 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:29.425820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:29.426661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:29.427296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:29.427976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:29.432796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:29.433291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:29.433773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:29.434550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:29.435097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:29.435548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:29.435860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:29.436089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:29.436684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:29.437344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:29.437843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:29.438320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:29.531722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:29.543757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ed156ba0-84b1-481b-8b86-f640f56d616a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.120551 Sep 19 16:18:29.544789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 354/669] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:29 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:29.617855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:29.627982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c338313d-f13b-49fd-b5af-4ad449dcdd58 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.207607 Sep 19 16:18:29.628576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 316/670] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:29 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:30.584514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:30.585331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:30.592780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:30.593278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:30.593719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:30.594627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:30.594919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:30.595347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:30.599942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:30.652700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:30.653419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:30.662956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:30.663638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:30.664239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:30.665042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:30.665765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:30.666730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:30.691020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:30.703711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ad9a451e-204e-43be-b230-334286330970 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.122792 Sep 19 16:18:30.704315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 355/671] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:30 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:30.859882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:30.869579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed552e14-3cd5-4345-9c22-05c9a1b604b5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.221483 Sep 19 16:18:30.870179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 317/672] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:30 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:31.736003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:31.736643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:31.745928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:31.746440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:31.746878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:31.747600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:31.748054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:31.749112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:31.823961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:31.832680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-da73b229-80b2-4d0f-964a-9f8ad8594906 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.101450 Sep 19 16:18:31.833274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 356/673] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:31 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:31.899438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:31.901711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:31.913087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:31.914304 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:31.915780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:31.917261 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:31.918287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:31.919287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:32.090064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:32.102797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-227b09df-90ef-4db3-8e95-90c7e5f41b74 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.211272 Sep 19 16:18:32.103827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 318/674] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:31 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:32.856543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:32.857593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:32.865512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:32.865997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:32.866953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:32.867679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:32.868142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:32.868570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:32.936880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:32.946347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4f4e95ab-afe9-4561-a394-b1474a537103 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.094196 Sep 19 16:18:32.947167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 357/675] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:32 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:33.140734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:33.141406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:33.148678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:33.149169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:33.149605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:33.150355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:33.150843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:33.151273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:33.301002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:33.309744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2ea2e3f1-cbe1-4ea4-aaa7-35d5650a1f1f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.172963 Sep 19 16:18:33.310337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 319/676] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:33 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:33.970058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:33.971202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:33.981312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:33.982195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:33.983008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:33.984349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:33.985152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:33.985935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:34.090106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:34.106517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6c5d73b7-1321-4d34-844d-341e3950d521 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.140494 Sep 19 16:18:34.110506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 358/677] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:33 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:34.334054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:34.334993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:34.344740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:34.345231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:34.345671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:34.346434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:34.346893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:34.347319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:34.504244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:34.519623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-247eb11e-42c5-453c-a627-40a97b4e5ed4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.189714 Sep 19 16:18:34.520666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 320/678] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:34 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:35.148872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:35.149884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:35.153030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:35.160040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:35.160634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:35.161282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:35.162089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:35.162765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:35.163276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:35.243918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:35.253618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0616ccf6-26b8-4cc6-8818-f65b05ece1ff tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.108988 Sep 19 16:18:35.254217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 359/679] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:35 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:35.559998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:35.560617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:35.562768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:35.563503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=10 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:35.571574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:35.572215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:35.572680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:35.574936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:35.574936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:35.574936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:35.726490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:35.737973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b0a234d-0ce8-4fe9-be63-d2f6aa5e9ed2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.181863 Sep 19 16:18:35.738603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 321/680] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:35 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:36.278772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:36.279588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:36.286773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:36.287322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:36.287796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:36.288556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:36.289104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:36.289681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:36.361570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:36.370804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3f8fcf8-bbec-4b7a-a4ba-0349a72faad7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.095919 Sep 19 16:18:36.371661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 360/681] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:36 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:36.779232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:36.780082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:36.792148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:36.793023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:36.793797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:36.796094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:36.796767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:36.797525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:36.967355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:36.978162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d607efb0-5212-4e89-8d1d-f6ff4f4db820 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.204870 Sep 19 16:18:36.978856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 322/682] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:36 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:37.396166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:37.396920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:37.404854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:37.405590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:37.406071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:37.407027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:37.407602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:37.408071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:37.480154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:37.489719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c9a58b4b-23ca-4cc7-b7fd-31d464066495 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.098271 Sep 19 16:18:37.490444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 361/683] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:37 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:38.003360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:38.004293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:38.011610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:38.012111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:38.012548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:38.013254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:38.013704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:38.014160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:38.182174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:38.195416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b9983cd3-59db-4341-b5e6-26f21b919fdd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.195891 Sep 19 16:18:38.196007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 323/684] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:37 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:38.588204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:38.589107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:38.596736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:38.597225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:38.597676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:38.598442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:38.598904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:38.599330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:38.672724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:38.682174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5989f8c7-36b5-4364-8b2b-e0a1b0575f0e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.170614 Sep 19 16:18:38.682930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 362/685] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:38 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:39.221350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:39.221994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:39.228740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:39.229217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:39.229674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:39.230396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:39.230844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:39.231263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:39.396100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:39.405929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d473e68e-fe2f-49c5-a1f4-c9ddaad3edd0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.188357 Sep 19 16:18:39.406590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 324/686] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:39 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:39.709951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:39.710727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:39.717362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:39.717911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:39.718652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:39.719577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:39.720151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:39.720709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:39.796361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:39.807530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-09d452a9-1942-484f-925f-36aea4f457b8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.104473 Sep 19 16:18:39.808125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 363/687] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:39 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:40.431465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:40.432335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:40.440298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:40.440790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:40.441232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:40.441956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:40.442451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:40.442888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:40.628134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:40.637708 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-22eab8a7-b661-4f68-b959-f7b6d2c7b309 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.210189 Sep 19 16:18:40.638314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 325/688] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:40 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:40.835591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:40.836702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:40.849058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:40.849864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:40.850669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:40.851895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:40.852773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:40.853514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:40.960993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:40.971184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e5ccdb52-df75-44cd-84d7-9f3c8667276e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.142642 Sep 19 16:18:40.971677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 364/689] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:40 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:41.662900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:41.663945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:41.673264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:41.673930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:41.674736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:41.675732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:41.676419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:41.676930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:41.846964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:41.856072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b49dd8a-1569-4178-922e-aa4a8a8608ce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.197937 Sep 19 16:18:41.856682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 326/690] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:41 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:41.995414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:41.996166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:41.999251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=64 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:42.005189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:42.005715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:42.006149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:42.006899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:42.007353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:42.007787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:42.089736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:42.098241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6d56ade4-f82e-4e29-818e-35f017881e06 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.106798 Sep 19 16:18:42.098780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 365/691] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:41 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:42.880721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:42.881346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:42.883937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=82,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:42.890080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:42.890611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:42.891048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:42.891763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:42.892227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:42.892651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:43.034785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:43.043549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-12a14cb7-785a-4aab-aa4c-fd09112dfe3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.166824 Sep 19 16:18:43.044109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 327/692] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:42 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:43.122051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:43.122728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:43.130071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:43.130601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:43.131038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:43.131743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:43.132200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:43.132621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:43.203534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:43.213925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-857e6d9b-aa0a-4210-a6da-b8521d1ac7b3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.095697 Sep 19 16:18:43.214771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 366/693] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:43 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:44.069662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:44.070306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:44.077610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:44.078119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:44.078576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:44.079305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:44.079756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:44.080181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:44.241143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:44.241459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:44.242781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:44.245098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:44.251981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-00168b09-4bac-4ecc-8cfc-ef8757511111 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.187112 Sep 19 16:18:44.252809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 328/694] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:44 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:44.256258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:44.257146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:44.257950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:44.259488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:44.260324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:44.261102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:44.358258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:44.366310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2343ef8d-809a-446a-a5c7-c3c025ad9d0d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.132277 Sep 19 16:18:44.366936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 367/695] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:44 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 134 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:45.277180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:45.277811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:45.285110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:45.285592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:45.286028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:45.286763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:45.287212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:45.287631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:45.397988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:45.398936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:45.406756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:45.407339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:45.407840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:45.408629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:45.409142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:45.409626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:45.468862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:45.478803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2a728981-c447-4a0a-a163-3c22863fbd2c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.205734 Sep 19 16:18:45.479584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 329/696] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:45 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:45.488846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:45.503981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-30dcbcd7-12c1-4f25-84fd-a4282b1d275f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.110226 Sep 19 16:18:45.505160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 368/697] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:45 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:46.514820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:46.516000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:46.523312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:46.523803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:46.524245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:46.524954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:46.525408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:46.525828 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:46.532489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:46.533626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:46.542442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:46.543115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:46.543836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:46.545050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:46.545648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:46.546429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:46.626032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:46.636110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-79fbe750-b6b6-463c-a813-1716a95703f8 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.110660 Sep 19 16:18:46.636705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 369/698] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:46 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:46.719876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:46.728562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-40bf0008-8f8d-41c3-bf64-2aa86e652e91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.217676 Sep 19 16:18:46.729148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 330/699] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:46 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:47.675184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:47.675864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:47.682872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:47.683362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:47.683814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:47.684540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:47.685000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:47.685998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:47.823471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:47.838028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f540188-a730-4d7a-b835-bac827d20076 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.168439 Sep 19 16:18:47.842552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 370/700] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:47 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:47.902997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:47.903877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:47.911010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:47.911412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:47.911828 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:47.912550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:47.913011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:47.913439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:48.149027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:48.157978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b5d9cbe5-13a6-43d0-a91d-c64dde264d10 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.408093 Sep 19 16:18:48.158641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 331/701] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:47 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:48.880126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:48.881014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:48.888108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:48.888596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:48.889043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:48.889838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:48.890298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:48.890861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:48.962188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:48.965813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:48.972207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0285397d-f5ac-47b2-ab90-82a8f10f4439 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.096251 Sep 19 16:18:48.972809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 371/702] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:48 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:49.181733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:49.182356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:49.189768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:49.190246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:49.190705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:49.191412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:49.191865 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:49.192295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:49.347615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:49.356521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-31ad2ce9-19ce-4670-b61c-2df357a4621f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.178613 Sep 19 16:18:49.357148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 332/703] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:49 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:49.995462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:49.996148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:50.003568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:50.004054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:50.004488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:50.005500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:50.005959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:50.006405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:50.082937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:50.091024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b348fc7e-4a29-44e1-b876-ad30bc84e578 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.099409 Sep 19 16:18:50.092311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 372/704] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:49 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:50.381406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:50.382460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:50.390545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:50.391267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:50.391855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:50.396209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:50.397539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:50.398583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:50.552884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:50.562629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fc214947-c361-451b-96af-74b79f7d7e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.186277 Sep 19 16:18:50.563405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 333/705] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:50 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:51.114446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:51.115067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:51.122131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.122643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.123075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.123774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.124238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.124665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.193698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:51.207562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-98075686-cf02-4434-b74e-d493c3d5ef8d tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.096702 Sep 19 16:18:51.208598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 373/706] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:51 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:51.373986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:51.374547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:51.381049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.381544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.382352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.383333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.384014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.384460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.418127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-40d274c0-832c-4b5b-8826-688ac794c352 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments/41f99edb-cead-4e63-be1b-662fbf00b32c" status: 200 len: 194 microversion: 2.20 time: 0.182663 Sep 19 16:18:51.418758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 334/707] 10.222.0.57 () {60 vars in 1454 bytes} [Mon Sep 19 16:18:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments/41f99edb-cead-4e63-be1b-662fbf00b32c => generated 194 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:51.503916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:51.504412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:51.511307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.511790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.512221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.512929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.513891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.514315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.593497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:51.594679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:51.606173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.607003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.607754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.608874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.609636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.610344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.630475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lazy-loading 'services' on Instance uuid 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:18:51.731075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2580e6f5-89c1-401a-bb95-0d6def972763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments/41f99edb-cead-4e63-be1b-662fbf00b32c" status: 202 len: 0 microversion: 2.20 time: 0.230598 Sep 19 16:18:51.732705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 374/708] 10.222.0.57 () {60 vars in 1457 bytes} [Mon Sep 19 16:18:51 2022] DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments/41f99edb-cead-4e63-be1b-662fbf00b32c => generated 0 bytes in 233 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:18:51.751901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:51.756544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:51.769093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.769920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.770718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.772006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:51.772826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:51.773478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:51.783380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:51.792054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5d80fa0a-7825-4774-aee5-4ec3a8123ff0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.206828 Sep 19 16:18:51.792644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 335/709] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:51 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:51.829365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-db11488b-fc5a-46ec-9e82-accf2deff86b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.085349 Sep 19 16:18:51.830396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 375/710] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:52.234141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:52.234786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:52.241075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.241569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.242003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.242758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.243212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.243652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.325913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:52.334333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-efef613d-80eb-4d82-b769-f7efd67a6dc0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.104522 Sep 19 16:18:52.346275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 336/711] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:52 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:52.818102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:52.818798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:52.825606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.826907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.827341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.828043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.828492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.828915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.847389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:52.850006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:52.857721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.858221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.858780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.859517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:52.859951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:52.860393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:52.919172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-849c98a9-b5b5-4749-9e52-b752d5ea83e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.075452 Sep 19 16:18:52.920235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 337/712] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:53.000359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:53.009200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-818007d4-45eb-4cf0-aa2a-109d5480d00c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.195077 Sep 19 16:18:53.009828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 376/713] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:52 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:53.372981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:53.373703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:53.380309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.380807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.381310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.382001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.382491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.382924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.458265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:53.467550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c833a992-789e-4088-9edf-d0efef69b89b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.099007 Sep 19 16:18:53.468487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 338/714] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:53 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:53.581685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:53.583940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:53.590428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.590915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.591364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.592217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.592671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.593098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.625418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f284d52-b61d-4b43-90e4-20d62dfb0d91 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.047750 Sep 19 16:18:53.625992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 377/715] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:18:53 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93/os-volume_attachments => generated 25 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 19 16:18:53.699410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:53.699909 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:53.707975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.708468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.708903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.709615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.710073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.710539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.739966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:18:53.839307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-629d6d05-f44e-4548-8eb7-0e51d7687a5c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 204 len: 0 microversion: 2.1 time: 0.143337 Sep 19 16:18:53.839888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 339/716] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:18:53 2022] DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:18:53.853072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:53.854142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:53.866407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.867293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.868250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.869539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.870345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.871111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.933692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:53.936246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:53.943400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.944189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.944630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.945359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:53.945816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:53.946267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:53.983500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4054d060-209b-4272-9587-379708b818cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054168 Sep 19 16:18:53.985580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 340/717] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:18:54.038754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:54.039921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:54.051620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:54.052526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:54.053303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:54.054566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:54.055401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:54.056184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:54.060334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:54.070694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-324ddde7-3ec5-4588-8277-49b99a45b2f9 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.224690 Sep 19 16:18:54.071769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 378/718] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:53 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:54.242002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:54.255028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-302470c7-a849-4c09-baab-04a8fbc39f8c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.223770 Sep 19 16:18:54.255625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 341/719] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:54 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:54.496789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:54.498429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:54.510982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:54.511617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:54.512396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:54.513730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:54.514729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:54.515522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:54.606999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:54.616074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-630c2ea1-0804-4318-b382-e3db06aabede tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.127142 Sep 19 16:18:54.616687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 379/720] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:54 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:55.000350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:55.002685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:55.009656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.010153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.010629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.011340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.011793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.012217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.047431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a804e499-0225-4a55-844e-ba4dab2751e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051002 Sep 19 16:18:55.047922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 342/721] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:55.099261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:55.099933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:55.107055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.107547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.107985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.108716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.109177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.109607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.257845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:55.267054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ea01ec4f-5f6c-4367-bf46-e4f142c18260 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.171810 Sep 19 16:18:55.267686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 380/722] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:55 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:55.292647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:55.293327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:55.300626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.301086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.301603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.302403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.302910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.303384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.491701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:55.495401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:55.502753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8f96538b-cd57-4e54-bdd5-58190483e16b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.214200 Sep 19 16:18:55.503400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 343/723] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:55 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:55.640051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:55.640674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:55.647418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.647910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.648350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.649146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:55.649602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:55.650229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:55.738058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:55.746628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-78b60b4b-25ae-422f-b52f-2f054155e546 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.111007 Sep 19 16:18:55.747402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 381/724] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:55 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:56.062158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:56.064884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:56.072276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.072766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.073203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.073920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.074444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.074868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.107713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c8addb81-591c-4504-b8bc-0e16bdd89625 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051072 Sep 19 16:18:56.108295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 344/725] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:56.293114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:56.293730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:56.300718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.301207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.301639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.302341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.302815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.303237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.448752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:56.457855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-74c79e82-e745-4ac6-9dc0-96de73585b17 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.168406 Sep 19 16:18:56.458493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 382/726] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:56 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:56.527905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:56.528654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:56.535306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.535792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.536220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.536927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.537383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.537806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.703839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:56.713025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-96523e5c-096e-4c24-ac9d-801c7f7a335c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.188965 Sep 19 16:18:56.713638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 345/727] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:56 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:56.771745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:56.772359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:56.778946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.779441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.779876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.780617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:56.781077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:56.781501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:56.847397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:56.860147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9ecbaef1-e65b-454b-8a06-10d2f8c94795 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.092288 Sep 19 16:18:56.860788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 383/728] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:56 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:57.120518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:57.122734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:57.129755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.130197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.130674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.131380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.131822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.132240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.166531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-41591a47-a7c9-4654-80c5-abee75d71a4b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050232 Sep 19 16:18:57.167119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 346/729] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:57.482241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:57.482969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:57.489849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.490337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.490821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.491549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.492002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.492437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.652169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:57.660425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b483fa12-258f-40df-9f37-d01b13cd7220 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.182107 Sep 19 16:18:57.661009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 384/730] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:57 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:57.736519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:57.737051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:57.743372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.743855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.744284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.744987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.745459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.745859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.891583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:57.892700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:57.899904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.900395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.900833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.901556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:57.902009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:57.902463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:57.911422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:57.923271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7f52be7a-64ab-4568-9d14-f615130c4d91 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.190649 Sep 19 16:18:57.924127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 347/731] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:57 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:57.992841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:58.001078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fac009dc-3958-4e7d-8097-c89652472fa4 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.116781 Sep 19 16:18:58.001666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 385/732] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:57 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:58.184676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:58.186902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:58.193553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.194054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.194511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:58.195231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.195678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.196095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:58.230019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-00385bda-ad46-4d29-a687-72015dc8710a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049289 Sep 19 16:18:58.230752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 348/733] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:18:58.684268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:58.685411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:58.692044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.692526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.692960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:58.693683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.694134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.694589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:58.847450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:58.856284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a7847584-5b56-4e58-9c2d-e2a2d1d74742 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.175903 Sep 19 16:18:58.857653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 386/734] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:58 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:58.949873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:58.950588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:58.956902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.957404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.957832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:58.958579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:58.959033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:58.959450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.025087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:59.025988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:59.034918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.035615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.036215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.037305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.038977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.039681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.121349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:59.128338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:18:59.134203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:18:59.134741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d05419c9-4aa8-44e0-8dfc-cfe9571504fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.188807 Sep 19 16:18:59.135736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 349/735] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:58 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:18:59.144072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bf9b589e-d6a6-4f31-a966-cdc7c2a694f3 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.123225 Sep 19 16:18:59.145129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 387/736] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:59 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:18:59.243751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:59.245887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:59.252371 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.252852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.253283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.253969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.254443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.254885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.288725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-29d3e381-86ff-44f3-912a-da2563df90ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048810 Sep 19 16:18:59.289237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 350/737] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:18:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:18:59.886222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:18:59.887393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:18:59.899974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.900858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.901590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:18:59.902904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:18:59.904260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:18:59.905067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.055993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:00.064773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d8241f9f-ba47-4255-aa9e-c10af2bca172 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.186114 Sep 19 16:19:00.065513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 388/738] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:18:59 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:00.163508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:00.164123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:00.169045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:00.170700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:00.171418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.171910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.172342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.173301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.173492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.173921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.179575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.180388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.180973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.181934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.182532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.183081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.266604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:00.275128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c8099f8c-a87f-4a13-8e87-9cc2782aaf6c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.110535 Sep 19 16:19:00.275726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 389/739] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:00 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:00.306224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:00.310127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:00.320821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.321678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.322448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.323305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:00.323945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:00.324673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:00.333896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:00.343527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d7e9d984-97f8-4162-b114-e0fc586adf5d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.183856 Sep 19 16:19:00.344392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 351/740] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:00 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:00.372574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7f7b8d21-7218-47c5-8246-8505a137a6bc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.073563 Sep 19 16:19:00.373638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 390/741] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:01.089779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:01.090400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:01.097229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.097767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.098244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.099037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.099530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.100058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.245655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:01.254544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f1c1c1e7-81d9-4e41-a658-33dc7e1c07e3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.168781 Sep 19 16:19:01.255172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 352/742] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:01 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:01.314792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:01.315615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:01.322249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.322765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.323198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.323903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.324352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.324770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.370822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:01.371646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:01.379796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.380649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.381388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.382210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.382888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.383387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.398755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:01.413301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-57d7328a-6481-4fd9-8bbd-13490d7eb087 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1245 microversion: 2.1 time: 0.102651 Sep 19 16:19:01.414312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 391/743] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:01 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1245 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:01.423483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:01.427811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:01.437196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.437852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.438333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.439224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:01.439728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:01.440190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:01.481615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7bb72248-894d-4be8-b296-a6504f2cf93a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065646 Sep 19 16:19:01.484630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 392/744] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:01.567146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:01.576183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-03e374d8-c86e-4643-a4b8-fd9dd9e00cb2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.210069 Sep 19 16:19:01.576796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 353/745] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:01 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:02.281256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:02.281977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:02.290735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.291395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.291985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.292807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.293387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.293942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.440441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:02.441181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:02.449160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.449748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.450262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.451669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.452148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.452584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.483191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:02.496231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b8b2528e-eb43-484e-b099-64e2b8460c3d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.220217 Sep 19 16:19:02.496841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 393/746] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:02 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:02.502778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:02.505102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:02.512259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.513090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.513531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.514339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.515628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.515628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.554620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:02.567001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fdc991f8-2565-4aa2-b904-7ba02ea5fc59 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061291 Sep 19 16:19:02.567001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 394/747] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:02.567459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-45cca2ea-41b5-40be-999b-91e6c6c5815b tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1237 microversion: 2.1 time: 0.127435 Sep 19 16:19:02.567459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 354/748] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:02 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1237 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:02.600935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:02.601551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:02.608633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.609123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.609559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.610267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:02.610756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:02.611182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:02.762681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:02.773831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6e733a38-1ea6-4e47-a148-7d1b67f5cebf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.176817 Sep 19 16:19:02.774476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 395/749] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:02 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:03.531860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:03.532524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:03.539821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.540617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.541396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.542263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.542787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.543255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.578776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:03.581929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:03.589826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.590311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.590784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.592427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.603042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.603042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.640967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3ab6f372-2a4b-41af-a950-a895cc6413f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067172 Sep 19 16:19:03.641549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 396/750] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:03.646586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:03.647493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:03.654430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.654928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.655361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.656074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.656530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.656967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.720810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:03.732945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-506d6728-9346-4f29-b87a-6130d0a0e697 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.206794 Sep 19 16:19:03.733718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:03.742901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 355/751] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:03 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:03.743206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dee16214-b8d6-488c-8fa7-25fbaf7d9730 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1237 microversion: 2.1 time: 0.099051 Sep 19 16:19:03.743206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 397/752] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:03 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1237 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:03.802335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:03.802979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:03.809463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.809972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.810415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.811127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:03.811583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:03.812008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:03.997754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:04.020157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-65f743b8-b565-4f54-8923-05fe75de68d7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.221375 Sep 19 16:19:04.042265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 356/753] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:03 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:04.659589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:04.662513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:04.672086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.672765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.673300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.674207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.674833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.675360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.712111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cb1a7c55-e7b7-4907-9db0-a8734dabe1ca tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058822 Sep 19 16:19:04.712701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 398/754] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:04.759113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:04.760157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:04.769917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.770681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.771152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:04.771382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.772008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:04.772208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.772843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.773501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.783261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.784021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.784628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.785923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:04.786500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:04.787077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:04.875680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:04.887327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4151f3de-966d-4d71-8662-b2206cabf302 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1293 microversion: 2.1 time: 0.122083 Sep 19 16:19:04.900203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 399/755] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:04 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1293 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:04.946857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:04.955643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-162bdc25-3b50-4343-a22b-58494bd07fa8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.200790 Sep 19 16:19:04.956277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 357/756] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:04 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:05.065918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:05.066667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:05.073575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.074062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.074541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.076043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.076504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.076933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.235197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:05.244747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d1f928d4-ebef-4f44-870b-c292c199cba7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.182901 Sep 19 16:19:05.245438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 400/757] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:05 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:05.735505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:05.739789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:05.752612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.754662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.755490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.756838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.757937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.759038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.827691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-07238f34-a8df-4510-b9be-a34651c2e740 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.101159 Sep 19 16:19:05.834219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 358/758] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:05.916135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:05.917454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:05.933593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.934863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.935775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.937239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.938532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.939427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:05.984015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:05.986544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:05.997587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:05.998484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:05.999281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.000603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.001449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.002203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.080853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:06.105084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3ea7a9fb-fb5f-4d59-a5de-787dd75622f6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1293 microversion: 2.1 time: 0.195346 Sep 19 16:19:06.106974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 401/759] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:05 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1293 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:06.290733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:06.291727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:06.318957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.320156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.321616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.323322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.324464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.326554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.584819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:06.599327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-59fbbe22-35af-466f-af4a-afe68b1f1333 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.323034 Sep 19 16:19:06.600280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 402/760] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:06 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:06.650796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:06.651625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:06.660096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.660431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.660957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.661822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.662409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.663030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.864619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:06.873716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-24be3771-384e-4a6d-ad8b-5e9722c1de6c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1673 microversion: 2.1 time: 0.895495 Sep 19 16:19:06.874428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 359/761] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:05 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1673 bytes in 897 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:06.885959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:06.888988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:06.896270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.896773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.897215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.897951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:06.899152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:06.899688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:06.921887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:06.951454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-30355eb2-a5ce-4d47-ab34-ea80abae267c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074773 Sep 19 16:19:06.953564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 360/762] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:07.018286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:07.033924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b2fe2fe0-238f-47ff-b686-7525477cc1f8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1595 microversion: 2.1 time: 0.387443 Sep 19 16:19:07.034968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 403/763] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:06 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1595 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:07.149289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:07.151359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:07.165669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.167376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.168582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:07.171474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.172712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.173730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:07.315460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:07.330860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b14e4214-04c9-4429-809b-0e4915554ffd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1299 microversion: 2.1 time: 0.188956 Sep 19 16:19:07.339228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 361/764] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:07 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1299 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:07.644938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:07.646129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:07.658117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.659468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.660450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:07.662007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.662727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.663666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:07.694605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:07.907157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:07.913631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:07.939880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.941279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.946795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:07.951481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:07.956577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:07.959114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.128787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:08.145505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-86c8ffb7-e7fd-4234-be94-4bb4275df0c2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1605 microversion: 2.1 time: 0.248234 Sep 19 16:19:08.147556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 362/765] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:07 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1605 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:08.158052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:08.162176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:08.175743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.177113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.178418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.180083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.182310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.183472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.256404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e3ea65a-709b-4b2c-957b-66a6a9fbbdbb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.106103 Sep 19 16:19:08.258358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 363/766] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:08 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:08.267685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "bae51cb6-fc0c-4343-9cd7-c04ef65d6b93", "name": "network-vif-deleted", "tag": "84b6192c-6cd6-4692-a397-69b77dc536c9"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:08.286535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.287885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.288955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.313015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] Creating event network-vif-deleted:84b6192c-6cd6-4692-a397-69b77dc536c9 for instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 on hv-cinder-85837 Sep 19 16:19:08.325765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-e58042f8-a3a5-44fc-bff6-6864a08331b2 req-ad756030-06f0-4c55-8d5f-ec9ab1a170c2 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064354 Sep 19 16:19:08.327418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 364/767] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:08.328891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:08.337898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:08.338951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:08.340225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-57b88a7f-9d65-4b66-b6db-f2789beba3a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1735 microversion: 2.1 time: 0.702501 Sep 19 16:19:08.359323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 404/768] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:07 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1735 bytes in 723 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:08.362840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.363744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.368905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.372003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.373970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.376952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:08.377581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.378054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:08.434259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.435165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.435901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.437329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.444816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.447076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.465480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:19:08.668950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9ae962ba-babf-4324-aa99-a9d1f20dddca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 204 len: 0 microversion: 2.1 time: 0.337993 Sep 19 16:19:08.677131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 365/769] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:19:08 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:19:08.688580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:08.689863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:08.702702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.703507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.704214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.705398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:08.706151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:08.706894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:08.847134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:08.861694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0130e9fa-fa94-4e97-984b-b4c40b4c27fd tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1299 microversion: 2.1 time: 0.492841 Sep 19 16:19:08.866602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 405/770] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:08 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1299 bytes in 499 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:09.000496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:09.024912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed737904-c427-4e7d-9d80-7835dd239268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1579 microversion: 2.1 time: 0.344900 Sep 19 16:19:09.040256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 366/771] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:08 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1579 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:09.185960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:09.188470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:09.209850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.211608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.212733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.213877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.215056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.217106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.276840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:09.280429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:09.300602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.301021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.301529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.302512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.303279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.303794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.316266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.316803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.317487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.318937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lazy-loading 'fault' on Instance uuid bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:19:09.380594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-65da6bbe-98ec-4213-ac5e-4e15c78a1687 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.110352 Sep 19 16:19:09.386596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 367/772] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:09 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:09.401389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:09.404082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:09.422039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:09.429545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.435952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.437127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.438772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.439943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.443694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.455232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-510d74ee-8ec1-41a7-94a9-6fe3b3569abb tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1326 microversion: 2.1 time: 0.278545 Sep 19 16:19:09.457901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 406/773] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:09 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1326 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:09.652625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:09.668686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cf19e119-8d24-4fb5-bded-14f2bdfc062f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1715 microversion: 2.1 time: 0.284680 Sep 19 16:19:09.670278 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 368/774] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:09 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1715 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:09.761377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:09.763409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:09.785698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.786628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.788006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.789789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.790921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.792041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.904065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:09.905224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:09.918700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.920507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.921554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.922908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:09.923766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:09.924541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:09.988955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Cannot 'detach_volume' instance fbfe137a-fc97-409e-9473-71a345427ae2 while it is in vm_state rescued Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:19:09.995273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:19:09.998096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Cannot 'detach_volume' instance fbfe137a-fc97-409e-9473-71a345427ae2 while it is in vm_state rescued Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance fbfe137a-fc97-409e-9473-71a345427ae2 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:19:09.999381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance fbfe137a-fc97-409e-9473-71a345427ae2 while it is in vm_state rescued Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance fbfe137a-fc97-409e-9473-71a345427ae2 while it is in vm_state rescued Sep 19 16:19:10.001693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:10.007758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 409 to user: Cannot 'detach_volume' instance fbfe137a-fc97-409e-9473-71a345427ae2 while it is in vm_state rescued {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:19:10.017402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2c8b8831-f51e-44c8-8c0d-e47db59f2aad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 409 len: 152 microversion: 2.1 time: 0.263392 Sep 19 16:19:10.037828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 407/775] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:19:09 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 152 bytes in 285 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 19 16:19:10.063304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:10.065796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:10.093859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.099305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.100406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.101700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.102532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.103292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.217587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b23022c4-7353-4d22-bb68-d2a46714bb87 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action" status: 202 len: 0 microversion: 2.1 time: 0.172537 Sep 19 16:19:10.221467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 408/776] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:19:10 2022] POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action => generated 0 bytes in 177 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:19:10.224969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:10.225731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:10.234033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.234692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.234988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.236157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.237214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.238153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.550273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:10.566501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-752f96bc-ea53-4f01-a9e9-5ca33a9992a0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.671101 Sep 19 16:19:10.568411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 369/777] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:09 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 674 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:10.579116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:10.580973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:10.624927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:10.646652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.647890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.648578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.649746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.650598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.656184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d6733fcd-a594-4d71-92b7-2cfa23bc97ba tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1579 microversion: 2.1 time: 0.435828 Sep 19 16:19:10.661481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.670628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 409/778] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:10 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1579 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:10.696816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:10.708494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:10.727203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.728526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.729295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.730481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.731175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.732416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.821988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8a0a1524-d409-42a0-b3c6-dfcf532cb4ba tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.148489 Sep 19 16:19:10.840988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 410/779] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:10 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:10.852582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:10.853638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:10.866965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.867874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.869223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.870033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.871231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.872033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.930546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:10.931672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:10.932461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:10.933460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lazy-loading 'fault' on Instance uuid bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:19:11.127085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:11.152116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-84b6b241-0d5c-405e-b8eb-c747e4e608cf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 200 len: 1326 microversion: 2.1 time: 0.309510 Sep 19 16:19:11.160098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 411/780] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:10 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 1326 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:11.271969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:11.298273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d5eb3c2f-6672-4706-be7b-972b9361f8f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1723 microversion: 2.1 time: 0.727504 Sep 19 16:19:11.300093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 370/781] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:10 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1723 bytes in 731 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:11.566565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:11.598821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:11.603066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:11.610925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=98,nova_api:UPDATE=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:11.614003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:11.623505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:11.627408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:11.651070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:11.652505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:11.653598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:11.657697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:11.659397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:11.660516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:11.679562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] Creating event network-changed:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:19:11.701491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-675a7a2e-a400-4e98-ae3d-8094415d6ee0 req-534fbdec-8d06-4999-96ba-b959e467d34b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.154960 Sep 19 16:19:11.708623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 412/782] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:11 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:11.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:11.737205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:11.755197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:11.763803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:11.769122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:11.772109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:11.776269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:11.778862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:11.781717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.319825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:12.338712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7f878277-2313-4d80-9dc6-956dda18cbce tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1579 microversion: 2.1 time: 0.623510 Sep 19 16:19:12.339664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 413/783] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:11 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1579 bytes in 626 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:12.346569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:12.348887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:12.357443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.357939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.358564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.359405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.359866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.360297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.400205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b14c9cd8-2b12-4507-b43c-862914f3abcc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059267 Sep 19 16:19:12.405662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 414/784] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:12.406566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:12.407136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:12.416359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.416849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.418785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.420696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.422641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.423376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.442202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 HTTP/1.1 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:19:12.445321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:19:12.447149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'bae51cb6-fc0c-4343-9cd7-c04ef65d6b93'} Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:19:12.448244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:19:12.450912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:19:12.454452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:19:12.455718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4471429b-4fa5-4613-94ff-af9c10f21f14 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 404 len: 111 microversion: 2.1 time: 0.053659 Sep 19 16:19:12.456812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 415/785] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:12 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 111 bytes in 55 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:19:12.462332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:12.463773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:12.472177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.472960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.474111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.475141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:12.475805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:12.476542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:12.646499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:12.677187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bc71784d-9dcd-43e4-b172-6b932ea863de tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 1.086549 Sep 19 16:19:12.678545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 371/786] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:11 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 1089 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:12.688538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-de85589a-cc5b-40af-9a1d-6c777d5e39ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '.version_select at 0x7fadbfb41ca0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:12.727643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-de85589a-cc5b-40af-9a1d-6c777d5e39ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-132643020" status: 202 len: 0 microversion: 2.1 time: 0.046650 Sep 19 16:19:12.728899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 372/787] 10.222.0.57 () {58 vars in 1218 bytes} [Mon Sep 19 16:19:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-132643020 => generated 0 bytes in 49 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:19:12.778545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:12.799709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eedd6ef8-7404-45aa-9ce3-1cbb93271e30 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1723 microversion: 2.1 time: 0.341309 Sep 19 16:19:12.801141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 416/788] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1723 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:13.399208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:13.400378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:13.429826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.430147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.435463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.437129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.440554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.444433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.529838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a34c2a71-df97-4e4d-ab68-da770b22a036", "tag": "5bdbe7af-21f9-4fe2-8859-99f053e3c9e2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:13.545916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.547350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.547994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.563041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] Creating event network-changed:5bdbe7af-21f9-4fe2-8859-99f053e3c9e2 for instance a34c2a71-df97-4e4d-ab68-da770b22a036 on hv-cinder-85837 Sep 19 16:19:13.570362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-4a2e2ef2-1360-4074-b7aa-6a13e2be1c57 req-3bd7ad11-dce5-460d-b49c-3f63a07147b7 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.245140 Sep 19 16:19:13.571694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 373/789] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:13.585289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:13.589179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:13.600105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.600790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.601342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.602239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.603715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.603715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.656943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-661be72b-beaa-4ae1-8ca9-8753d1c2fffa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083314 Sep 19 16:19:13.657948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 374/790] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:13 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:13.718359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:13.723850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:13.729176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:13.744439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.746201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d0ab3185-ab25-4826-9434-d4784a8c94a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1511 microversion: 2.1 time: 0.353610 Sep 19 16:19:13.747660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.747958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 417/791] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:13 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1511 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:13.749895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.754731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.755521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.755982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.826144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:13.826966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:13.842094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.842957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.843588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:13.847235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:13.848068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:13.848715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:14.616383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:14.651923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-12c8bb5a-6c4a-4b3e-890f-727ce5d4a0d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1723 microversion: 2.1 time: 0.829358 Sep 19 16:19:14.653022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 418/792] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:13 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1723 bytes in 832 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:14.676475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:14.679919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:14.704368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:14.705195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:14.705798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:14.706578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:14.707718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:14.708155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:14.738457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:14.743497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c61cece4-1b21-407a-8c8b-16dfbf8b7d62 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.075914 Sep 19 16:19:14.744671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 419/793] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:14.769906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3fd46c8c-c13e-46cd-b503-f0bfc3b653d6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 1.038039 Sep 19 16:19:14.771723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 375/794] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:13 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 1052 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:14.791693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:14.792878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:14.799926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:14.800685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:14.801348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:14.802443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:14.803163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:14.803760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:14.957773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:14.962172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:14.968599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7cc3aeb2-30f6-430c-98fe-ed0a7f985c9b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.180678 Sep 19 16:19:14.969774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 420/795] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:14 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:15.097746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-686217570"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:15.098633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:15.099371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-08451633-a7d9-4e00-819d-4375bf3bc3a2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'create', calling method: .version_select at 0x7fadbf844040>, body: {"keypair": {"name": "tempest-keypair-2001211246", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBIVT2dP+SL/DV9uBVw4mNmHxJymLz5BLOxFx9tv4bymNJBurfJrR3thj9iajz1QkbxLRDLxZedRdkO0dxTSWHAy9LC1jQKLwanquzDWLpOFkXVhEHFM8+nZTZjRIEt9VoQ=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:15.106049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:15.106969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:15.107733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:15.108646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:15.109263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:15.109922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:15.123433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-08451633-a7d9-4e00-819d-4375bf3bc3a2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:19:15.128856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-08451633-a7d9-4e00-819d-4375bf3bc3a2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for user 754c1b5cb47746cf93c52d6dc0b8fa01 and project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:19:15.163542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-08451633-a7d9-4e00-819d-4375bf3bc3a2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.068908 Sep 19 16:19:15.164591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 421/796] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:19:15 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:15.406015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.network.security_group_api [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Adding security group 50e8421a-4912-47e0-b522-134e07eb3ef6 to port 96c941aa-fa1b-4d66-97da-985273910cfe Sep 19 16:19:15.619157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "f5c5064d-5dce-44c2-8610-006177ddd542", "name": "network-vif-deleted", "tag": "d6454427-db60-426c-9f7f-ef69e803c8b5"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:15.639609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:15.641798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:15.643174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:15.667452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] Creating event network-vif-deleted:d6454427-db60-426c-9f7f-ef69e803c8b5 for instance f5c5064d-5dce-44c2-8610-006177ddd542 on hv-cinder-85837 Sep 19 16:19:15.677110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-8d6aa14e-05cd-4266-903d-cb3da63c7a76 req-ca1937ea-fb42-4592-893e-268c5c512d92 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064672 Sep 19 16:19:15.678226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 422/797] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:15.688894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:15.690123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:15.702455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:15.703700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:15.704769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:15.705984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:15.706900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:15.707547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.012650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-354a55fc-0fed-4124-ba2b-38ea3abc9fec tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action" status: 202 len: 0 microversion: 2.1 time: 0.930201 Sep 19 16:19:16.013869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 376/798] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:19:15 2022] POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action => generated 0 bytes in 932 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:19:16.031618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:16.042432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:16.064522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.065844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.067280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.068971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.070541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.071842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.138824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:16.162641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9bc30b12-0f01-48ea-b944-44ebb91618cd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1729 microversion: 2.1 time: 0.482002 Sep 19 16:19:16.164817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 423/799] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1729 bytes in 486 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:16.165536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5316da5d-7c1e-4761-91bc-d63698d7f0f5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.149449 Sep 19 16:19:16.173727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 377/800] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:16.177979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:16.180176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:16.186074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:16.187899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:16.194128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.194679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.201798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.204431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.205200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.205379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.207227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.207778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.207961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.208334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.208757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.209002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.219592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:16.382167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:16.406510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0516108e-a0c8-4bb2-8f55-33135b17ff74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.235855 Sep 19 16:19:16.413859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 378/801] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:16 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:16.436681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:16.439853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:16.463572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.475677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.476911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.478592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.479759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.480772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.495582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:16.513325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-677a4cb3-c75b-4e88-aac3-a00c2072c2b2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.336285 Sep 19 16:19:16.515961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 424/802] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:16 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:16.526540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:16.527926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:16.542558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.544535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.546913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.548593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:16.549903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:16.551002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:16.707338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e10128ee-a974-433c-b49b-ed0cd54410f4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.188651 Sep 19 16:19:16.707971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 425/803] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:19:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-interface => generated 284 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:17.191380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:17.198661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:17.218991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.220288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.221957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.225543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.226948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.229583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.317239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b5650016-49b7-493d-88be-9189b25c23e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.134416 Sep 19 16:19:17.319209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 426/804] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:17 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:17.460349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:17.461099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:17.478555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.480006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.480408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:17.482348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.486296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.487895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.489088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.493484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c032a500-4c56-47ad-b781-13bd0989466c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1760 microversion: 2.1 time: 1.076866 Sep 19 16:19:17.494164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 379/805] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1760 bytes in 1079 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:17.542031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:17.543005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:17.555928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.556444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.556892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.557636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.558100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.558570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.667986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:17.689332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7710558a-c838-432b-a5ff-efd233f28436 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.234282 Sep 19 16:19:17.691031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 427/806] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:17 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:17.723333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:17.742208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.743445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.749046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.773078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] Creating event network-changed:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:19:17.797991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-18f9b573-84db-4efb-8770-e650da113ec4 req-1e93fa01-2854-468c-a50e-fb5976c88a72 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087071 Sep 19 16:19:17.799615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 428/807] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:17.860115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:17.871053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:17.889326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.891581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.892579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:17.893982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:17.894962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:17.895886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.049730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:18.061247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a85397a-46be-4b18-aab2-b55db4780c0f tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.523361 Sep 19 16:19:18.072396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 380/808] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:17 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 535 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:18.075726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-47cc60c3-2456-48bd-bdb3-e82d2040db50 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action" status: 202 len: 0 microversion: 2.1 time: 0.269976 Sep 19 16:19:18.080269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 429/809] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:19:17 2022] POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action => generated 0 bytes in 280 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:19:18.120089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:18.123116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:18.143903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.144947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.145886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.158940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.162574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.163552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.340271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:18.343488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:18.352324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.353127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.353795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.354953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.356619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.360285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.403826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-33a0508d-9308-4032-aff1-53cba348b6f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.071223 Sep 19 16:19:18.406826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 430/810] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:18.671090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:18.681281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c7aabe3b-eb11-4dda-a93b-e894b8e580d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1752 microversion: 2.1 time: 0.589535 Sep 19 16:19:18.682625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 381/811] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:18 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1752 bytes in 593 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:18.699447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:18.719228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.720085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.720880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.728995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:18.730028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:18.737992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.738917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.739621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.740801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:18.741556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:18.742218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:18.744837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] Creating event network-changed:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:19:18.766011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-635b6ec3-8b90-4bc4-9401-5d9283c5208e req-1deafda4-08d7-412a-83bb-d0f9f2ee1f0e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.204178 Sep 19 16:19:18.767134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 431/812] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:18 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:18.910020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:18.922456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e8c83fc3-2cef-4178-a331-cdc14bb68851 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.197141 Sep 19 16:19:18.923113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 382/813] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:18 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:19.123603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:19.125069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:19.138114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.139673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.140615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.142253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.143320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.144238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.430568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:19.438851 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:19.454965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:19.464862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d423168b-08b7-453d-ab99-5a5147c284c7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.350093 Sep 19 16:19:19.474458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.475816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.476998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.478441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.479342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 432/814] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:19 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:19.479468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.480221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.485743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a34c2a71-df97-4e4d-ab68-da770b22a036", "name": "network-vif-plugged", "status": "completed", "tag": "5bdbe7af-21f9-4fe2-8859-99f053e3c9e2"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:19.509498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.509980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.510616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.525715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] Creating event network-vif-plugged:5bdbe7af-21f9-4fe2-8859-99f053e3c9e2 for instance a34c2a71-df97-4e4d-ab68-da770b22a036 on hv-cinder-85837 Sep 19 16:19:19.532582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-61101a51-c451-4910-a990-cf57580ea30e req-dd12bfca-2bca-4f05-ac47-1ecc0b274aa5 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.051583 Sep 19 16:19:19.533764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 433/815] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:19.555891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-09f0725a-b804-4578-be73-466fcbe2c096 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.142038 Sep 19 16:19:19.557410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 383/816] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:19.724757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:19.725389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:19.733773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.734720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.735311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.736383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:19.737076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:19.737921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:19.899514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-32653075", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "3002445e-0f83-4b0b-b1fa-f136b28412de"}], "security_groups": [{"name": "tempest-securitygroup--1124654500"}], "key_name": "tempest-keypair-2001211246", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:20.000226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:20.011269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7a629271-1355-4cb0-a8a8-f90a891de1a5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1752 microversion: 2.1 time: 0.307060 Sep 19 16:19:20.012936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 434/817] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:19 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1752 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:20.018463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:20.019556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:20.027086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.027848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.029456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.030515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.031317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.031985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.141794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:20.151969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b8a849d3-4f6f-4b5b-ad52-34469c4a387d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.137803 Sep 19 16:19:20.152783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 435/818] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:20 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:20.440547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] validate_networks() for [('3002445e-0f83-4b0b-b1fa-f136b28412de', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:19:20.497908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:20.498973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:20.506521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.507217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.507779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.508620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.509111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.509675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.678994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:20.688263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b6160287-04dd-4288-948d-f8cc1a1a6deb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.195079 Sep 19 16:19:20.689223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 436/819] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:20 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:20.694186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:20.696525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:20.703521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.704001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.704614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.705511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.706118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.706743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.745823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-66be2d90-1ad6-486b-8d04-2f12d84ee58d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055279 Sep 19 16:19:20.747404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 437/820] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:20.787747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:20.799643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:20.800485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:20.801161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:20.816941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] Creating event network-changed:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:19:20.825166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-88e9477e-e33b-4fa8-9dad-0ca6cfe73bc6 req-3b672908-2d16-41d4-b5e0-849f5e6bc086 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040624 Sep 19 16:19:20.826224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 438/821] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:20 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:21.061698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:21.062919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:21.074830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.075873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.076781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.078092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.079034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.079810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.164603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:19:21.165246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:19:21.165659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:19:21.166148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:19:21.166681 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:19:21.182165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.183817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.183966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.185366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.186362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.187462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.223361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:19:21.233061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Getting quotas for user 754c1b5cb47746cf93c52d6dc0b8fa01 and project 23c705bc001e4228bdfcdfa916a368e9. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:19:21.256539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:19:21.260010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:19:21.328877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-495d2f77-fb9d-489d-b888-61c442d801ee tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.438168 Sep 19 16:19:21.336528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 384/822] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:19:19 2022] POST /compute/v2.1/servers => generated 402 bytes in 1447 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:19:21.342853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:21.343483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:21.350559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.351056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.351497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.352220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.352680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.353113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.353305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:21.369582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c58d4bb6-31fd-4567-a470-4bd86c6610d2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1752 microversion: 2.1 time: 0.316467 Sep 19 16:19:21.371100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 439/823] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:21 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1752 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:21.395009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-720567ec-976e-45fc-a738-539069e3ee86 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:21.396111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-720567ec-976e-45fc-a738-539069e3ee86 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:21.477230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-720567ec-976e-45fc-a738-539069e3ee86 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:21.486502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-720567ec-976e-45fc-a738-539069e3ee86 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1236 microversion: 2.1 time: 0.113883 Sep 19 16:19:21.487888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 440/824] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:21 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1236 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:21.533274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:21.550745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8462d6b3-c99c-4e81-8cd7-89551395c757 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.212498 Sep 19 16:19:21.551653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 385/825] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:21 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:21.728393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:21.729266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:21.736858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.737358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.737799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.738575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.739053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.739511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.767234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:21.777051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:21.784596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.785196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.785734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.786621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:21.787128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:21.787664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:21.833420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-61b5f841-4bad-4fd1-a8f3-d0fc52d620be tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.073550 Sep 19 16:19:21.834235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 386/826] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:21.920891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:21.936800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e4981fb4-6bf0-484d-a0bb-fba6a82e7dd1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1367 microversion: 2.1 time: 0.212994 Sep 19 16:19:21.938490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 441/827] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:21 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1367 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:22.419101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:22.419671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:22.426732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.427220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.427657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.428376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.428890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.429311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.510733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:22.511471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:22.519129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.520050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.520715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.521689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.522220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.522914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.528748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:22.602623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:22.619046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8c97b5c8-07cf-45cf-bffc-4e87387d655f tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1288 microversion: 2.1 time: 0.111144 Sep 19 16:19:22.620205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 442/828] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:22 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1288 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:22.629203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:22.630399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:22.642826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.643974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.644842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.646190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.647325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.648087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.755114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:22.769901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e4c65157-143f-4346-91cf-232548a73b3e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.147918 Sep 19 16:19:22.771121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 443/829] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:22 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:22.832663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:22.846938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6ca5e14a-f577-44fb-a16e-4837766b0293 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.433309 Sep 19 16:19:22.848507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 387/830] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:22 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:22.856812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:22.859940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:22.868382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.869437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.870180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.871389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.872196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.872896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.926183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5719471b-32c4-4d6e-8b87-d1260aa60b3f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.081324 Sep 19 16:19:22.927066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 444/831] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:22.975993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:22.976606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:22.983478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.983963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.984394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:22.985102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:22.985548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:22.985968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.131651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:23.140235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ccbdfcbf-ceb7-4dda-8f8d-3f65633ec7e9 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1557 microversion: 2.1 time: 0.168036 Sep 19 16:19:23.140823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 388/832] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:22 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1557 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:23.179121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:23.180609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:23.187731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.188227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.188667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.189396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.189861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.190579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.288423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5769f1e7-5ee7-4776-b990-41cf44afb9dc tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.113548 Sep 19 16:19:23.290021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 445/833] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:19:23 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036/os-interface => generated 285 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:23.662180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:23.663765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:23.676234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.677075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.677885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.679269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.680118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.680900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.798952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:23.800037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:23.801113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:23.814291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-94e27f2b-ac35-4b51-93cd-6b6725d54b42 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1294 microversion: 2.1 time: 0.157959 Sep 19 16:19:23.815676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.816886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.817876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.819685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.820827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.821916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.830933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 389/834] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:23 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1294 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:23.877506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:23.878998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:23.889904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.891199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.891980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.893502 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.894614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.895389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.932057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:23.944849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-385743c5-5e0b-4649-ae42-01eb4847e3ab tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.154575 Sep 19 16:19:23.945981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 446/835] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:23 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:23.951631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:23.954174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:23.961777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.962660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.963318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:23.964242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:23.964892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:23.965560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:24.217366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:24.227771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5001731f-8de8-418f-81ef-6d306113ce6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.355568 Sep 19 16:19:24.229408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 390/836] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:23 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 358 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:24.274125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5866db03-d92c-463d-823d-15b9954a5ede tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.326229 Sep 19 16:19:24.275279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 447/837] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:24.746132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a34c2a71-df97-4e4d-ab68-da770b22a036", "tag": "5bdbe7af-21f9-4fe2-8859-99f053e3c9e2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:24.771493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:24.772430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:24.773430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:24.797075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] Creating event network-changed:5bdbe7af-21f9-4fe2-8859-99f053e3c9e2 for instance a34c2a71-df97-4e4d-ab68-da770b22a036 on hv-cinder-85837 Sep 19 16:19:24.807252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-9d15f2ec-9232-49e5-b0d3-b5dc7075e4f3 req-1f71be15-6455-4968-a28b-4243424a762b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068383 Sep 19 16:19:24.810122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 391/838] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:24.858279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:24.859415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:24.871125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:24.872562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:24.873337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:24.874641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:24.875720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:24.876697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:24.987725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:24.988909 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:24.996257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:24.997411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:24.998182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:24.999266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:25.000021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:25.001056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:25.125680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:25.141300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b790f52b-eee5-4ce7-8df4-0ee3aefb96a1 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.290781 Sep 19 16:19:25.142549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 448/839] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:24 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:25.256113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:25.257205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:25.269401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:25.270296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:25.271410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:25.272337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:25.272841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:25.273879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:25.392247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:25.412403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-211cd7c5-5e67-49c9-a4ca-2c60edbe556f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.428993 Sep 19 16:19:25.440254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 392/840] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:24 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 459 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:25.458924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:25.467508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:25.489629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:25.491163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:25.493181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:25.496109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:25.497416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:25.499542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:25.566438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d9d7d377-cf61-49c0-b959-520b99a95c36 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.122251 Sep 19 16:19:25.568292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 393/841] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:25.616694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:25.636412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1d1994b4-560a-4207-9bfb-a7cc7d8a3b87 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.387091 Sep 19 16:19:25.638414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 449/842] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:25 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:26.185485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:26.186647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:26.208415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.209216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.209929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.211458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.212227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.212959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.443895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4af93002-f79c-48e0-9a49-a348edf588c8", "tag": "60258110-cac3-4864-a18c-34a601420154"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:26.466043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.467717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.468695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.489155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] Creating event network-changed:60258110-cac3-4864-a18c-34a601420154 for instance 4af93002-f79c-48e0-9a49-a348edf588c8 on hv-cinder-85837 Sep 19 16:19:26.495675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-c005fad1-f8b8-4cc6-a0ee-37d6653cf8cf req-3d172ff2-33b9-4cf9-9c53-62bfcd1d09f3 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063590 Sep 19 16:19:26.497327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 450/843] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:26.504176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:26.505243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:26.514823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.515486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.516026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.516851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.517578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.518744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.537463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:26.566474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-85252fbd-401a-458d-b057-b07b00001ef0 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.383225 Sep 19 16:19:26.567665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 394/844] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:26 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 390 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:26.595682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:26.602755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:26.611801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.613307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.614279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.615354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.616092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.616758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.653502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:26.665429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1963c55d-b747-4e78-b690-4b3c33cc8685 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076612 Sep 19 16:19:26.669199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 395/845] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:26 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:26.673272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7401f5de-fa6d-4210-9852-f215e6bf21c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.174027 Sep 19 16:19:26.674334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 451/846] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:26 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:26.675473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:26.676186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:26.685664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.686266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.687402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.688265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:26.688806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:26.689308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:26.694295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:26.963112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:26.973295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-54007a87-9926-4616-9646-75f1d410fc06 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.302591 Sep 19 16:19:26.974021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 396/847] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:26 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:27.608335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:27.608967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:27.615962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.616507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.616977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.617749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.618259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.618779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.680255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:27.683332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:27.692613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.693109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.693552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.694272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.694769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.695422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.747607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5406c843-f18e-4477-ac8f-d62a26e519b5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072288 Sep 19 16:19:27.749171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 397/848] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:27.758329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:27.759553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:27.770925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.772004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.772864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.774162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:27.775036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:27.775810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:27.863188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:27.872751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:27.875584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b5e55ab8-4343-4752-9568-42277f1f420c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.271172 Sep 19 16:19:27.876235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 452/849] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:27 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 273 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:27.884611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8567e559-7e57-48fe-bcaf-fee0644d02ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.133622 Sep 19 16:19:27.885425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 398/850] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:27 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:28.010058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:28.010811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:28.020203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.020981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.021648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.022770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.023386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.023977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.229144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:28.242850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6587e47f-971c-4e50-9309-b9ac544cc227 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.236319 Sep 19 16:19:28.243908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 453/851] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:28 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:28.763706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:28.766594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:28.773748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.774658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.775165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.776065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.776596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.777093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.821699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2d0e553e-78ba-4ba8-9f82-5f0da254306a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063268 Sep 19 16:19:28.822672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 399/852] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:28.905336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:28.907606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:28.915892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.916249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:28.916633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.917348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:28.917540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.918243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.919781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.920578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.929052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.929918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.930758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:28.932170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:28.932834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:28.933904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:29.039669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:29.054954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6b2ffe2c-22b9-4067-a94b-781bd114cd84 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.145570 Sep 19 16:19:29.056120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 400/853] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:28 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:29.107107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:29.117110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c600ecb7-a7b2-4323-8d74-29a7f73d21a8 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.216374 Sep 19 16:19:29.117695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 454/854] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:28 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:29.288092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:29.288895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:29.300404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:29.301223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:29.302215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:29.303281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:29.304134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:29.304913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:29.514872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:29.524163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fef9cb21-dfd1-4d6e-8a25-5bf2e0ed80f6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.243696 Sep 19 16:19:29.524760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 401/855] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:29 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:29.839916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:29.843270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:29.851099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:29.851835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:29.852384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:29.853267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:29.853838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:29.854408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:29.892603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-464c0e2c-f9e1-4716-95e2-bf53f2f31f51 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057425 Sep 19 16:19:29.893885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 455/856] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:30.101225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:30.102349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:30.114844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.115745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.116530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.117876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.118782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.119548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.141870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:30.142766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:30.150354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.151117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.151768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.152816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.153907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.154554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.222461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:30.239157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-47a352a7-7e58-405a-8fa6-183952b9a28d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.144487 Sep 19 16:19:30.240274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 402/857] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:30 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:30.347359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:30.359452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d4fd458e-3106-4603-9c55-b6af19e2f2bc tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.222027 Sep 19 16:19:30.360386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 456/858] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:30 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:30.553306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:30.554481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:30.566820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.567713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.568507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.569786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.570595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.571365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.781390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:30.798877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-50f02f05-2b55-4402-9a37-366e5d0b09f5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.252871 Sep 19 16:19:30.799971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 403/859] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:30 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 256 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:30.911220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:30.913627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:30.916312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=99,nova_api:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:30.922850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.923384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.923854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.924597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:30.925088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:30.925620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:30.978227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8d1acd72-aced-4f2c-8b6e-faef5e8a9bd7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.071204 Sep 19 16:19:30.979308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 457/860] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:31.278352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:31.280001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:31.284307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:31.285485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:31.299166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.300067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.300883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.302178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.303021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.303782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.412290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:31.421522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-59f9fcdb-778d-4207-908a-608132d5f11d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.150685 Sep 19 16:19:31.422483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 404/861] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:31 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:31.427536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:31.428145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:31.440105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.440511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4af93002-f79c-48e0-9a49-a348edf588c8", "name": "network-vif-plugged", "status": "completed", "tag": "60258110-cac3-4864-a18c-34a601420154"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:31.441348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.442236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.444236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.445076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.445878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.462091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.462620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.463055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.476355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] Creating event network-vif-plugged:60258110-cac3-4864-a18c-34a601420154 for instance 4af93002-f79c-48e0-9a49-a348edf588c8 on hv-cinder-85837 Sep 19 16:19:31.481778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-97073e82-13e2-47a6-a065-9d6dcdcd11e1 req-cf9fbf3e-db23-4a59-ac97-5aa97f0278ff service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.146567 Sep 19 16:19:31.482765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 458/862] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:31 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:31.616840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:31.628897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a4812bc0-16ae-4f6b-9c8e-7a52d2d0e0ca tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.205241 Sep 19 16:19:31.629553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 405/863] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:31 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:31.837030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=6 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:31.841385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:31.842099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:31.849624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.850284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.850759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.851478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:31.851931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:31.852357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:31.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:32.002248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:32.016217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.017118 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.017915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.019472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.020302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.021071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.038553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:32.050431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7a032bfa-4e52-4d47-b827-510b46a76596 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.214316 Sep 19 16:19:32.051029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 459/864] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:31 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:32.087354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6807d79b-2ab2-4085-ac15-f2e91c870f1c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.097417 Sep 19 16:19:32.088411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 406/865] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:32.460644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:32.462397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:32.472776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.473317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.473797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.476030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.476571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.477083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.499790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:32.548789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:32.557563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-936cca0c-4171-4485-9c2a-143307d65677 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.101575 Sep 19 16:19:32.558154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 460/866] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:32 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:32.653432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:32.654066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:32.661030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.661519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.661985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.662736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:32.663200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:32.663632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:32.817408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:32.828984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-52bbde9f-cdba-4cb3-ab0d-5ba3976fa988 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1362 microversion: 2.1 time: 0.180019 Sep 19 16:19:32.829575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 407/867] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:32 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1362 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:33.076749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:33.077668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:33.084444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.084930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.085365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.086073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.086592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.087018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.104379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:33.108042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:33.117761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.118599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.119314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.120490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.121243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.121944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.160543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b3cc339c-fd08-46d5-a4be-d17751a5908c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063312 Sep 19 16:19:33.161111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 408/868] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:33.283164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:33.291822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc99c161-fc88-4272-8f0f-1defed22b7a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.219988 Sep 19 16:19:33.292425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 461/869] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:33 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:33.581847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:33.582508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:33.589149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.589641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.590126 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.590984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.591446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.591878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.670286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:33.679159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a875fbf3-3abb-475a-8e84-459617d2c0fb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.101777 Sep 19 16:19:33.680050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 409/870] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:33 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:33.853497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:33.854222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:33.861658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.862148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.862632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:33.863346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:33.863801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:33.864280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.006748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:34.014935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-06a295ac-df88-4e8a-8ddd-67d9e78d33fe tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1548 microversion: 2.1 time: 0.165642 Sep 19 16:19:34.015529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 462/871] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:33 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1548 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:34.040103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:34.040882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:34.048837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.049378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.049808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.050567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.051032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.051474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.141409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-03acf9ec-02b6-47b8-8f88-a26b07ab3ddd tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.106496 Sep 19 16:19:34.141998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 410/872] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:19:34 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8/os-interface => generated 284 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:19:34.175469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:34.178997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:34.186469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.187017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.187451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.188191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.188642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.189060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.223637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1671516c-79b5-4478-923c-99d5f9dd2362 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052505 Sep 19 16:19:34.224426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 463/873] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:34.315740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:34.316361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:34.322817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.323307 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.323744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.324520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.325480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.325905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.623470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:34.633570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b2d60ec2-cb6e-4902-a907-4047ed7a616c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.321709 Sep 19 16:19:34.634179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 411/874] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:34 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 323 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:34.709258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:34.710065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:34.719006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.719638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.720163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.721006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:34.721560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:34.722093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:34.818709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:34.828322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8dc85625-dbe4-436a-b395-cbad86d8f4cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.123401 Sep 19 16:19:34.828958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 464/875] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:34 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:35.237298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:35.240269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:35.243637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=96,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:35.251627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.252487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.253143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.253983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.254633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.255155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.293672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-babb60d7-dec6-43bd-a2d6-d3eebb2ea52c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059939 Sep 19 16:19:35.294064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 412/876] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:35.627766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4af93002-f79c-48e0-9a49-a348edf588c8", "tag": "60258110-cac3-4864-a18c-34a601420154"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:19:35.639262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.639748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.640187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.656337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] Creating event network-changed:60258110-cac3-4864-a18c-34a601420154 for instance 4af93002-f79c-48e0-9a49-a348edf588c8 on hv-cinder-85837 Sep 19 16:19:35.664688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-e9328dce-c53d-43e9-9741-a043045b43c1 req-f99a6d3b-3b65-4878-89bc-6c87bca8d2c1 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.040298 Sep 19 16:19:35.665694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 465/877] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:19:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:19:35.683602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:35.684781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:35.694736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.695232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.695673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.696390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.696862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.697303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.702431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:35.858683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:35.859878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:35.872539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.873461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.874701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.876049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:35.877341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:35.878319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:35.967582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:35.987545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-09d7bcc1-6c4b-49e7-9388-cdd4e5b9fe58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.310923 Sep 19 16:19:35.988528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 413/878] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:35 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:36.009076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:36.022202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4669f7e7-13b7-41e5-ba46-11bcd08713cd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.170693 Sep 19 16:19:36.023764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 466/879] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:35 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:36.311495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:36.315503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:36.327446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:36.328349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:36.329156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:36.330500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:36.331376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:36.332166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:36.391414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad455844-15d1-474f-8ca1-b7df291759a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.086448 Sep 19 16:19:36.394882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 414/880] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:37.024473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:37.025163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:37.032493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.032975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.033402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.034113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.034610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.035029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.059819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:37.060753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:37.069098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.070159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.071075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.072445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.073154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.073976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.171064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:37.184905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b6ae8cd1-9242-4a83-93e3-ac63202a53b2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.130246 Sep 19 16:19:37.186024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 415/881] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:37 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 133 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:37.253773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:37.263862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ac73ad0-76b2-4706-a086-c63a0dc90211 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.243052 Sep 19 16:19:37.264501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 467/882] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:37 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:37.407176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:37.409375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:37.416520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.417016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.417462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.418773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:37.419290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:37.419716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:37.456745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e4b704d-db75-4787-9179-441a9cb2a23a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053679 Sep 19 16:19:37.457412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 416/883] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:38.209777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:38.211000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:38.220651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.221156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.221592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.222318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.222824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.223251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.295935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:38.296476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:38.296821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:38.303421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.303926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.304359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.305071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.305525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.305946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.311245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-467f535a-75e2-4ca0-a530-153c53493a35 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.106159 Sep 19 16:19:38.312312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 468/884] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:38 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:38.471054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:38.473655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:38.482091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.482766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.483408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.484419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:38.484926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:38.485393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:38.501816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:38.516425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4dec368e-4f75-40b1-ab33-0292d731f960 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.225111 Sep 19 16:19:38.517435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 417/885] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:38 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:38.530577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-adf45da7-bdd8-48ce-83f5-15afa9117bf9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064307 Sep 19 16:19:38.531311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 469/886] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:39.336728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:39.337256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:39.343517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.343997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.344429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.345127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.345578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.346001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.414180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:39.423483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f59c2848-4eed-4409-86c9-860c6bfe209c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090788 Sep 19 16:19:39.424074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 418/887] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:39 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:39.544574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:39.545426 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:39.546060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:39.547576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:39.553687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.554173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.554780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.555481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.555670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.555942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.556092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.556358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.556507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.557084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:39.557565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:39.557994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:39.617256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2537feab-5b28-41ed-ab1c-a5b5804a4b47 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076110 Sep 19 16:19:39.618183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 419/888] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:39.983653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:39.999424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-26b28f38-edb2-4aed-97d7-0c174fe69bf5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.458519 Sep 19 16:19:40.000509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 470/889] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:39 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:40.512845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:40.513925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:40.525346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:40.526139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:40.526897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:40.529073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:40.530144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:40.531031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:40.608877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:40.617645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2fe59087-e0f7-46f1-b845-8128c7ac0340 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.111927 Sep 19 16:19:40.618612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 420/890] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:40 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:40.633782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:40.636285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:40.643774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:40.644526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:40.645064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:40.645783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:40.646506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:40.647027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:40.686870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2bc167c7-3b01-442d-a2f0-638b5a2267f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056881 Sep 19 16:19:40.687458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 471/891] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:41.023882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:41.024478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:41.030843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.031253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.031689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.032396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.032847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.033272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.194419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:41.204030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b3e134ba-9900-41e5-a991-e831d6d52407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.184056 Sep 19 16:19:41.204729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 421/892] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:41 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:41.643928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:41.644561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:41.652877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.653361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.653798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.654558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.655012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.655460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.703471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:41.706531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:41.714773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.715290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.715724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.716441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:41.716896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:41.717328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:41.733381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:41.741847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9ebc3709-811e-4f88-a6dd-5ffad40bd301 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102427 Sep 19 16:19:41.742763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 472/893] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:41 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:41.755680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7083908b-c5d4-44c0-bed9-c72fe1403ca0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.059403 Sep 19 16:19:41.756266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 422/894] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:42.229453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:42.230058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:42.237895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.238394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.238841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.239543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.239989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.240407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.420666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:42.430874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e99241e5-5b9e-4606-9db8-d8f2193f0512 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.205395 Sep 19 16:19:42.431462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 473/895] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:42 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:42.769226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:42.770248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:42.773583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:42.777248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:42.782265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.783757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.783934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.785015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.785813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.786559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.789663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.790489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.791209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.792384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:42.793142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:42.793859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:42.838968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-79edcbe7-92ec-40a6-a70c-1f54847e2106 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.072486 Sep 19 16:19:42.839657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 474/896] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:42 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:42.874265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:42.889452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fce7d69d-3457-45b4-8826-159944464a6d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.127172 Sep 19 16:19:42.890519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 423/897] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:42 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:43.455335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:43.456160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:43.463573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.464100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.464577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.465472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.465961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.466460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.630440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:43.639436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5c5bddd8-007b-495d-82e1-50d9a22d7efd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.188893 Sep 19 16:19:43.640044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 475/898] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:43 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:43.856222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:43.858416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:43.865164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.865649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.866076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.866809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.867252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.867669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.899967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9f6398d0-725a-430a-a0c5-ca44e5808402 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048218 Sep 19 16:19:43.900555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 424/899] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:43.913550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:43.914312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:43.922074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.922864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.923415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.924233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:43.924811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:43.925335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:43.994852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:44.005469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eea4e552-be98-4bd9-95cf-6004d898e9fd tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.096068 Sep 19 16:19:44.006071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 476/900] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:43 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:44.664261 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:44.664965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:44.672655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:44.673289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:44.673855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:44.674680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:44.675185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:44.675737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:44.883934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:44.894785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a971ad4d-2772-4645-b0f0-0dd1be02f407 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1757 microversion: 2.1 time: 0.234612 Sep 19 16:19:44.895520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 425/901] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:44 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1757 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:44.917298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:44.921933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:44.934836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:44.935737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:44.936540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:44.937801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:44.938684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:44.939489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:44.977035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:44.986091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8bc51066-c5c6-421c-b843-c53de1ffb561 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.076576 Sep 19 16:19:44.987099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 477/902] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:45.029480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:45.030152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:45.037131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:45.037640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:45.038084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:45.038848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:45.039332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:45.039833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:45.112699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:45.121776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-275cbde8-1abc-478c-861d-eef624363b5d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.096357 Sep 19 16:19:45.122358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 426/903] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:45 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:45.920839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:45.921752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:45.928704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:45.929194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:45.929649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:45.930417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:45.930878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:45.931313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:46.001973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:46.005099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:46.014181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:46.014719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:46.015164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:46.015877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:46.016338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:46.016764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:46.057563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1269b5b7-237a-42ac-83e5-1abd09c9d409 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060340 Sep 19 16:19:46.058233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 427/904] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:46.109872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:46.118720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-78a40b70-becb-404a-9916-7e652ae7c8a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1760 microversion: 2.1 time: 0.202603 Sep 19 16:19:46.119309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 478/905] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:45 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1760 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:46.145828 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:46.146660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:46.153423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:46.154072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:46.154664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:46.155496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:46.156060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:46.157609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:46.227593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:46.236115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-48c30ac0-2bfa-4af6-ab64-132d4526f84c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094304 Sep 19 16:19:46.236720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 428/906] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:46 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:47.075904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:47.078127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:47.085681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:47.086170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:47.086639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:47.087354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:47.087807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:47.088225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:47.121789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-72fa234d-1e3a-4f43-9f6f-409183c7faac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050692 Sep 19 16:19:47.122407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 479/907] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:47.260361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:47.260986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:47.268093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:47.268594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:47.269035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:47.269765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:47.270228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:47.270705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:47.339898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:47.343724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:19:47.349146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6703bad1-80c1-4c33-8be6-729132781f14 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093142 Sep 19 16:19:47.349744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 429/908] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:47 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:48.135674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:48.138321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:48.150316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:48.151240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:48.152044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:48.153307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:48.154126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:48.154871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:48.192076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-53506d9f-bae3-455f-aec1-d7bba1a485e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061274 Sep 19 16:19:48.192642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 480/909] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:48.374563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:48.375191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:48.382026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:48.382556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:48.382999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:48.383717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:48.384175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:48.384611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:48.453819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:48.463781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7a03c5c8-b164-4b70-af0b-27c5937494bf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093822 Sep 19 16:19:48.464575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 430/910] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:48 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:49.204775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:49.207039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:49.214060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:49.215263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:49.215852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:49.216684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:49.217251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:49.217800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:49.261156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3926b8d8-a8bf-4d0d-bf4d-49ab982bb857 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060369 Sep 19 16:19:49.261896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 481/911] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:49.487091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:49.487714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:49.494306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:49.494826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:49.495258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:49.496000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:49.496444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:49.496864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:49.566620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:49.575935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d6634228-f895-4385-8b53-6bd24866acef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092733 Sep 19 16:19:49.576521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 431/912] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:49 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:50.274513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:50.276703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:50.283546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:50.284037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:50.284477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:50.285189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:50.285646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:50.286070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:50.319685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-99b61f52-f648-43f4-8abc-6166fef96103 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049205 Sep 19 16:19:50.320276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 482/913] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:50.599515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:50.600140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:50.607867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:50.608365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:50.608806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:50.609529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:50.609987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:50.610444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:50.678026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:50.687932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-44b0ca64-03ab-406e-a78f-2c405d554d34 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092263 Sep 19 16:19:50.688572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 432/914] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:50 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:51.336092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:51.338429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:51.345874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:51.346366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:51.346838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:51.347539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:51.347992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:51.348419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:51.382256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5728916b-b7a8-4fff-801d-de4454c7a1d7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050253 Sep 19 16:19:51.382895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 483/915] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:51.726751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:51.727506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:51.735565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:51.736089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:51.736532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:51.737244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:51.737698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:51.738127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:51.816456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:51.825901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f053ff85-a5fc-4e8e-acfa-52d077e0df98 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.103261 Sep 19 16:19:51.826545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 433/916] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:51 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:52.396114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:52.398293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:52.405212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:52.405701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:52.406137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:52.406882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:52.407358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:52.407781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:52.441777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-140687a6-66de-4e7e-9d80-c71f52557920 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049713 Sep 19 16:19:52.442364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 484/917] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:52.850472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:52.851242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:52.859419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:52.859970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:52.860444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:52.861306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:52.861812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:52.862280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:52.931271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:52.939782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-51503c8c-ac98-4b8c-817e-11f7c9a1fa4c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093431 Sep 19 16:19:52.940503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 434/918] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:52 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:53.459747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:53.462251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:53.471443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:53.472068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:53.472541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:53.473291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:53.473810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:53.474300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:53.512042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-68514f4a-6c6f-43ea-b8f4-0f93c045d92c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057588 Sep 19 16:19:53.512628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 485/919] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:19:53.963041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:53.963686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:53.970622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:53.971364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:53.971917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:53.972636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:53.973095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:53.973519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:54.070593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:54.078799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-025e111b-5078-4e75-8326-b964f985581f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.119638 Sep 19 16:19:54.079393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 435/920] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:54.526855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:54.530024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:54.537124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:54.537657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:54.538130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:54.538933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:54.539425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:54.539885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:54.586673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9257cc08-e7c2-4ab4-96ca-16315887ee49 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064405 Sep 19 16:19:54.587720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 486/921] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:55.108187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:55.108747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:55.115629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:55.116119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:55.116552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:55.117256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:55.117707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:55.118128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:55.187261 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:55.195800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b693860d-c85c-429f-a045-0cd1cff0f580 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091529 Sep 19 16:19:55.196394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 436/922] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:55 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:55.603344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:55.605568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:55.612475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:55.613040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:55.613537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:55.614420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:55.614969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:55.615555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:55.649075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-62c1e7a8-8408-460d-87a2-a7972caec3c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049566 Sep 19 16:19:55.649562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 487/923] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:56.219849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:56.220493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:56.227186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:56.227670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:56.228102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:56.228814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:56.229266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:56.229692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:56.310215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:56.318404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bdf615a9-da9f-421d-bc6c-bf791b140453 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102426 Sep 19 16:19:56.318984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 437/924] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:56 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:56.662243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:56.664510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:56.672178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:56.672666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:56.673102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:56.673819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:56.674281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:56.674741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:56.707516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-435258f0-f29a-466d-808f-ab7d3c971ac1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049286 Sep 19 16:19:56.708105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 488/925] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:57.342361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:57.343107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:57.349833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:57.350313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:57.350773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:57.351466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:57.351922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:57.352343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:57.424312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:57.432767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a91677bf-a47d-47fc-b7bb-9d35fb60d429 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094772 Sep 19 16:19:57.433394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 438/926] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:57 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:57.721585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:57.723985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:57.731086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:57.731714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:57.732202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:57.732953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:57.733444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:57.733907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:57.767607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fc86e4cc-5d40-48a0-b40c-c5b92425b375 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050091 Sep 19 16:19:57.768386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 489/927] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:58.456446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:58.457193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:58.463970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:58.464998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:58.465451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:58.466181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:58.466669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:58.467104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:58.537331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:58.545863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4d52d860-1391-4808-9bb6-3b7fa928e59e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093362 Sep 19 16:19:58.546739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 439/928] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:58 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:19:58.782620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:58.785357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:58.792648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:58.793147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:58.793585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:58.794317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:58.794804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:58.795233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:58.830272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a7901a60-e5cc-47b0-b7e0-0df30a5001ad tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052626 Sep 19 16:19:58.830902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 490/929] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:19:59.570167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:59.570925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:59.577696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:59.578184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:59.578713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:59.579744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:59.581078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:59.581523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:59.653200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:19:59.661773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bbb314f1-619d-4225-8d3b-3251b3df8479 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095531 Sep 19 16:19:59.662363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 440/930] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:19:59 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:19:59.843873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:19:59.846128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:19:59.852798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:59.853288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:59.853738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:59.854631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:19:59.855464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:19:59.855899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:19:59.889727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-72a9fa5c-0095-4103-9a81-982571039040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050230 Sep 19 16:19:59.890309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 491/931] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:19:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:00.685732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:00.686445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:00.694472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:00.695034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:00.695611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:00.696406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:00.696920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:00.697413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:00.770106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:00.782562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f1262d8e-813d-4637-a802-82454eb8376d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100669 Sep 19 16:20:00.783328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 441/932] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:00 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:00.904956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:00.907239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:00.915104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:00.915640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:00.916123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:00.916884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:00.917403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:00.917911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:00.952973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5e57d38a-a34c-4d9d-8e61-c06dfef2e487 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052840 Sep 19 16:20:00.953573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 492/933] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:01.295579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:01.296076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:01.302630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.303122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.303552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:01.304265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.304715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.305135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:01.331960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a5d3e0e0-a355-4c41-91eb-e098e4e68627 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036/ips" status: 200 len: 150 microversion: 2.1 time: 0.040361 Sep 19 16:20:01.332573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 442/934] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:20:01 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036/ips => generated 150 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:20:01.805420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:01.806028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:01.808007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=72 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:01.813797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.814281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.814748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:01.815463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.815946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.816365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:01.885702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:01.898913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cf424519-7082-449a-bb92-19272da2fb86 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.097190 Sep 19 16:20:01.899647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 493/935] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:01 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:01.966347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:01.968543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:01.977427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.978298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.979096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:01.980888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:01.981692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:01.982572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:02.029516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-59ee3a7a-5320-4329-afc6-7f647b6d309d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067390 Sep 19 16:20:02.030693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 443/936] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:02.923515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:02.924193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:02.931618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:02.932112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:02.932548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:02.933309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:02.933763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:02.934188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:02.954039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:02.999405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:03.007657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-890118cb-081f-4c85-9868-831ddb157598 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088571 Sep 19 16:20:03.008241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 494/937] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:02 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:03.043257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:03.045405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:03.052271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:03.052753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:03.053191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:03.053892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:03.054340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:03.054804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:03.088014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c38cf5d1-55f7-4763-8053-c0c867b5d40a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048710 Sep 19 16:20:03.088594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 444/938] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:04.049513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:04.050098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:04.056592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:04.057078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:04.057511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:04.058215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:04.058720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:04.059150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:04.101391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:04.103607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:04.111482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:04.111979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:04.112515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:04.113241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:04.113703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:04.114137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:04.130132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:04.137973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8ed71dbb-9719-45cf-bcac-39aadd8e57e9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092749 Sep 19 16:20:04.138904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 495/939] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:04 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:04.146610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6c9efd8a-7412-440c-b199-3f488e722459 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049181 Sep 19 16:20:04.147204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 445/940] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:05.160363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:05.161494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:05.162515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:05.163063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:05.171306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:05.172148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:05.172425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:05.172794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:05.173027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:05.173528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:05.173772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:05.174317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:05.174540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:05.174984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:05.175198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:05.175625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:05.212003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e0011398-868d-4a09-ae7b-67e4b5b3ce33 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056214 Sep 19 16:20:05.213430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 496/941] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:05.245136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:05.250184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:05.255726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d7f1fdb6-5187-4343-9859-ab1f6cf1cd3f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098440 Sep 19 16:20:05.256471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 446/942] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:05 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:06.226526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:06.228982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:06.240182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:06.241053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:06.241824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:06.243117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:06.243909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:06.244658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:06.279333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:06.280138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:06.282778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=70 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:06.288370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:06.288871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:06.289314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:06.290027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:06.290517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:06.290948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:06.306662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7f7c78b7-f483-4e35-8eb8-a08ea0a69e16 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083669 Sep 19 16:20:06.308325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 497/943] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:06.373149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:06.382553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-de05e621-a5b9-460d-be89-e3d5e73574ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.106626 Sep 19 16:20:06.383466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 447/944] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:06 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:07.329839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:07.333930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:07.347536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:07.348782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:07.349800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:07.351421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:07.352330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:07.353351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:07.408888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:07.409655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:07.416955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:07.418504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:07.419045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:07.419993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:07.420574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:07.421106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:07.427591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d375b538-cd62-4ca6-8d16-9aa7ee05e5a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.105816 Sep 19 16:20:07.428642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 498/945] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:07.488396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:07.497389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d96f002f-6dd5-4c8b-af5c-00f70b1e0d43 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092650 Sep 19 16:20:07.498180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 448/946] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:07 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:08.442470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:08.444734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:08.452124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:08.452615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:08.453061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:08.453772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:08.454229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:08.454691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:08.488801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0fc29b2c-8f10-4c15-abbe-214f5369b55d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050227 Sep 19 16:20:08.489299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 499/947] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:08 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:08.521092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:08.521706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:08.528509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:08.528998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:08.529439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:08.530145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:08.530632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:08.531057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:08.601697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:08.609616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-52be9cb6-f25d-4d7f-9d87-39f02961a791 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092442 Sep 19 16:20:08.610187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 449/948] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:08 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:09.502212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:09.504957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:09.512317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:09.512797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:09.513228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:09.513922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:09.514413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:09.514830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:09.545921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e1b61ec1-b893-4b0a-b649-f8276b6e9657 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047878 Sep 19 16:20:09.546809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 500/949] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:09 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:09.633443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:09.634041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:09.641077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:09.641562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:09.641993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:09.642764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:09.643206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:09.643630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:09.711791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:09.721544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d9f6720f-d34c-47b0-9053-8c326d8a3523 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091816 Sep 19 16:20:09.722406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 450/950] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:09 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:10.559872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:10.562196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:10.569108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:10.569644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:10.571100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:10.571100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:10.571407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:10.571764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:10.606025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-46aedb3b-ee22-4f87-8900-b5ef8fe2f86f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050777 Sep 19 16:20:10.606715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 501/951] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:10 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:10.746196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:10.746834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:10.753872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:10.754408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:10.754847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:10.755574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:10.756034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:10.756466 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:10.838106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:10.847569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9143e7ca-6fef-4152-8b8a-92762e581548 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.105355 Sep 19 16:20:10.848164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 451/952] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:10 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:11.619405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:11.622683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:11.630052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:11.630581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:11.631023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:11.631757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:11.632213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:11.632641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:11.674332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d167eb7f-cca9-4319-9388-595429d4c4fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058903 Sep 19 16:20:11.675021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 502/953] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:11 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:11.872239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:11.872891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:11.880035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:11.880530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:11.880973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:11.881699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:11.882159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:11.882640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:11.951403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:11.960354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1a21421a-4122-49b6-a6c9-76c7321c8dca tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092705 Sep 19 16:20:11.961018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 452/954] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:11 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:12.691235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:12.694037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:12.701705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:12.702305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:12.702841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:12.703755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:12.704396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:12.705028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:12.741257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9556890-d58b-47d3-a541-7ace7fcaf0fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054331 Sep 19 16:20:12.741876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 503/955] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:12.985514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:12.986366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:12.993711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:12.994241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:12.994779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:12.995636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:12.996305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:12.996824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:13.066291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:13.074684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-88b4619d-187b-4b5a-8d90-78c7a7b1ef76 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093624 Sep 19 16:20:13.075283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 453/956] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:12 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:13.757508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:13.760295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:13.768873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:13.769528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:13.770041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:13.770830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:13.771339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:13.771834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:13.807186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-40815f7e-52ee-4de7-a44e-f9a19041eead tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053898 Sep 19 16:20:13.808272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 504/957] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:13 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:14.098137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:14.098980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:14.106216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:14.106731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:14.107169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:14.107883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:14.108335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:14.108753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:14.178009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:14.186970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-69cb6b23-c709-4f4d-a8d3-aa775ec5c7ea tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092609 Sep 19 16:20:14.187641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 454/958] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:14 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:14.820209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:14.822505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:14.829958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:14.830351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:14.830812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:14.831509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:14.831957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:14.832369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:14.866076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-13e69252-16da-403c-9b91-0e29782c08fb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049905 Sep 19 16:20:14.866713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 505/959] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:15.210727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:15.211375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:15.218987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:15.219660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:15.220181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:15.221007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:15.221596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:15.222131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:15.292931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:15.305255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-70df7d90-e1e6-466f-b949-7020a513468a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098245 Sep 19 16:20:15.306434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 455/960] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:15 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:15.882025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:15.884576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:15.891923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:15.892463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:15.893007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:15.893762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:15.894269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:15.894772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:15.927592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b1d59777-17cf-413c-953b-7e2ba3fbf988 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050538 Sep 19 16:20:15.928168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 506/961] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:16.330405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:16.331870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:16.342434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:16.343016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:16.343623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:16.344449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:16.344957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:16.345458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:16.415684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:16.424824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-26bedc89-f4a3-48a4-971d-1c6fc13c580f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098685 Sep 19 16:20:16.425426 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 456/962] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:16 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:16.942339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:16.944784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:16.953439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:16.954078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:16.954662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:16.955504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:16.956087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:16.956667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:16.995294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b34d70b6-ff98-42d8-be98-a4747c7d96a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058021 Sep 19 16:20:16.996275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 507/963] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:17.457871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:17.458939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:17.465850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:17.466346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:17.466826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:17.467549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:17.468005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:17.468434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:17.536419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:17.544877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c28ce0d7-00dc-4d23-91e6-2069e322cbd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091173 Sep 19 16:20:17.545476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 457/964] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:17 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:18.012983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:18.015492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:18.022628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:18.023193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:18.023753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:18.024601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:18.025111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:18.025569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:18.059144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-805a3510-a194-459a-8c1d-b46a63f39d8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050575 Sep 19 16:20:18.059718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 508/965] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:18.570956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:18.571906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:18.579531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:18.579975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:18.580298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:18.581041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:18.581508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:18.581936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:18.660628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:18.669624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-74daed04-356c-423a-af87-aadda7c75010 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102725 Sep 19 16:20:18.670347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 458/966] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:18 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:19.072584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:19.074878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:19.082292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.082897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.083385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.084159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.084740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.085444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.124159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4cd2f93e-07d8-4824-bd6a-8623b2f9ee64 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055615 Sep 19 16:20:19.124760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 509/967] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:19.500734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:19.501231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:19.507651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.508144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.508609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.509327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.509801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.510443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.535403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7783f783-18e9-45cd-98db-780715592d35 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8/ips" status: 200 len: 146 microversion: 2.1 time: 0.038556 Sep 19 16:20:19.535997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 459/968] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:20:19 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8/ips => generated 146 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:20:19.693504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:19.694192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:19.700642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.701148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.701580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.702284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:19.702772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:19.703197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:19.773005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:19.788112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-28d79cb3-1fb3-4cd5-972d-2bd13325a25e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098226 Sep 19 16:20:19.789188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 510/969] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:19 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:20.138044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:20.140262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:20.147342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:20.147839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:20.148282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:20.149021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:20.149482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:20.149912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:20.183984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e48ecf07-8d3d-4da7-aa0d-4354b00748e7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050680 Sep 19 16:20:20.184595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 460/970] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:20.830393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:20.831083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:20.838053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:20.838797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:20.839283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:20.840121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:20.840627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:20.841136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:20.846974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:20.924008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:20.932954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-619faabf-e86e-4bdb-801b-ad4b76a135d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.106050 Sep 19 16:20:20.933567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 511/971] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:20 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:21.197147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:21.199301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:21.207389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:21.207646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:21.208077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:21.208773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:21.209361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:21.209674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:21.242770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b1e150ff-867a-4f81-b950-f4e648e019d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049740 Sep 19 16:20:21.243368 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 461/972] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:21.957245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:21.957907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:21.965943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:21.966524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:21.967009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:21.967763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:21.968257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:21.968731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.047522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:22.058682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-058bb8e7-caea-49d7-91b9-fa39b38a359d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.105388 Sep 19 16:20:22.059293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 512/973] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:21 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:22.075187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-686217570"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:20:22.075970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:22.084661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:22.085402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:22.086098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.087210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:22.088482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:22.089549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.253712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.network.security_group_api [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Removing security group 50e8421a-4912-47e0-b522-134e07eb3ef6 from port 96c941aa-fa1b-4d66-97da-985273910cfe Sep 19 16:20:22.260367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:22.264154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:22.271529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:22.272029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:22.272465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.273189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:22.273638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:22.274060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.321170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8c75c0f8-ca19-4e6d-9eb9-1b754070a21a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.067476 Sep 19 16:20:22.330531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 513/974] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:20:22.671162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5555822b-080b-4f1e-8398-af12bbe1dffd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action" status: 202 len: 0 microversion: 2.1 time: 0.600480 Sep 19 16:20:22.677546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 462/975] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:20:22 2022] POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action => generated 0 bytes in 603 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:20:22.687720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:20:22.706256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:22.707323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:22.708216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:22.743966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] Creating event network-changed:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:20:22.758197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-1caea129-75b9-4b68-aca8-350409ea0b6f req-996eb82c-bd33-4f41-830c-73af3c91a54a service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076777 Sep 19 16:20:22.760009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 514/976] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:20:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:20:23.090111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:23.091014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:23.097793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:23.098302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:23.098992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:23.099701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:23.100140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:23.100570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:23.204318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:23.218417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3fe0a1e5-8e28-45f3-b7d7-32a37353fd41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.132267 Sep 19 16:20:23.225360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 463/977] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:23 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:23.340132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:23.343362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:23.350135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:23.350659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:23.351092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:23.351808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:23.352255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:23.352674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:23.408346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c1a04537-cbc8-4926-ac7f-0d6552f9fc61 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074728 Sep 19 16:20:23.409432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 515/978] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:24.210537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:24.212323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:24.212680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:20:24.224579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.225422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.226151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.227385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.228185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.228959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.231612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.232751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.233597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.237873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:24.261882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] Creating event network-changed:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:20:24.271395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-9f239eb9-edd8-4a12-8809-762548d1f68a req-c178495b-1261-413f-a9b5-851482448da7 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.064398 Sep 19 16:20:24.273533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 516/979] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:20:24 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:20:24.283665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:24.285649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:24.293825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.294746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.295402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.296423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.297274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.297810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.393956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:24.410744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ef4041d9-a49d-439c-83c0-6137b5b289de tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.134883 Sep 19 16:20:24.411804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 517/980] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:24 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:24.433329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:24.438185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:24.451136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.451930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.452718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.454311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:24.455197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:24.455978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:24.519047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3456a04c-82a2-4901-a356-2d7df3bf8fbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.093834 Sep 19 16:20:24.520070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 518/981] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:24 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:25.435950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:25.436758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:25.444066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:25.444558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:25.445006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:25.445728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:25.446181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:25.446651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:25.528157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:25.537569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2414e414-688b-46d1-b5fc-0a8b764f4458 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.106490 Sep 19 16:20:25.538621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 519/982] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:25 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:25.545011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:25.547894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:25.555511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:25.556236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:25.556843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:25.558358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:25.559042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:25.559585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:25.591540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-956366f7-00ff-41de-9b0a-c1bcdb980a56 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051753 Sep 19 16:20:25.592129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 520/983] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:26.567304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:26.568385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:26.575559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:26.576043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:26.576480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:26.577200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:26.577658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:26.578085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:26.647009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:26.656307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0ecf7e96-68f3-4b68-9d07-939a3abe3247 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092944 Sep 19 16:20:26.656901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 521/984] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:26 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:26.661879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:26.664230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:26.670953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:26.671430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:26.672126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:26.672840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:26.673320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:26.673719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:26.708305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c8ec5821-3db1-437e-8d7b-6f0520b835b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050322 Sep 19 16:20:26.708888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 522/985] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:26 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:27.684618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:27.685537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:27.696189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:27.697040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:27.697789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:27.699099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:27.699925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:27.702272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:27.772624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:27.787208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bf28711d-aacb-466e-af1a-09add9ef52d0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.109035 Sep 19 16:20:27.788083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 523/986] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:27 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:27.797613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:27.801408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:27.813047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:27.813866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:27.814695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:27.815930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:27.816769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:27.817532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:27.868697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-521c564c-88c0-45a6-bfbb-9a3c35044852 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.079320 Sep 19 16:20:27.869289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 524/987] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:28.813506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:28.814263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:28.821127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:28.821614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:28.822049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:28.822814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:28.823269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:28.823746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:28.911946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:28.924686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc545bcd-ec16-4b94-8ca0-0fb4cfb5fdbf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.114981 Sep 19 16:20:28.925619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 525/988] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:28 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:28.948723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:28.951494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:28.959504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:28.960126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:28.960652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:28.961661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:28.962233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:28.962870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:28.998456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f32a1850-ce64-4391-b3fe-922070c61a13 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054803 Sep 19 16:20:28.999047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 526/989] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:29.952580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:29.953284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:29.959899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:29.960393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:29.960831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:29.961544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:29.961999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:29.962449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:30.031736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:30.040494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e9189563-a6ab-49a7-870f-9bd52056179b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092191 Sep 19 16:20:30.041169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 527/990] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:29 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:30.050339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:30.054205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:30.065170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:30.066043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:30.066855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:30.068122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:30.068944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:30.069724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:30.104370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7ac7563e-f76c-4136-aa89-42f336801360 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061712 Sep 19 16:20:30.104970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 528/991] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:31.082317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:31.083537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:31.096234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:31.097136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:31.097917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:31.099282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:31.100094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:31.100865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:31.110277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:31.207558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:31.218505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fcd8fadf-5d6d-4f3a-ad2e-d24132b02221 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.143304 Sep 19 16:20:31.219582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 529/992] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:31 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:31.228752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:31.233040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:31.240675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:31.241156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:31.241585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:31.242323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:31.242803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:31.243265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:31.276505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c031e8a6-4d5f-4622-944b-bf741ad5f9a0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054983 Sep 19 16:20:31.277088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 530/993] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:32.243000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:32.243865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:32.245957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:20:32.246130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:20:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 (ip 10.222.0.57) !!! Sep 19 16:20:32.247680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=76 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:32.256364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:32.256850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:32.257287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:32.257999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:32.258487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:32.258911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:32.333733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:32.345172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ae44aa38-71cd-48ba-8170-20011c62be90 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.105934 Sep 19 16:20:32.345882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 531/994] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:32 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:32.351997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:32.354669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:32.362101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:32.362709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:32.363210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:32.363994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:32.364502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:32.364984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:32.400205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dff74019-c2fb-4651-a30e-99fccbe0bad8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052930 Sep 19 16:20:32.401636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 532/995] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:33.371542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:33.372279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:33.379345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:33.379827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:33.380266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:33.381015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:33.381473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:33.381901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:33.578744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:33.587329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fa5c3012-3d12-4c42-b7d3-e50a85305cfc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.219938 Sep 19 16:20:33.587926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 533/996] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:33 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:33.593119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:33.595461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:33.602303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:33.602839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:33.603281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:33.604527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:33.604985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:33.605409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:33.641322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-700fedaf-5559-42c0-b64a-54e2d557ba5a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051954 Sep 19 16:20:33.641804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 534/997] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:34.221277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=35 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:34.267138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:34.617210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:34.617854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:34.625558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:34.626048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:34.626522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:34.627240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:34.627726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:34.628190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:34.702408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:34.712773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb7c0360-10a1-4e7e-9738-b45019891c69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099806 Sep 19 16:20:34.714321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 535/998] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:34 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:34.722642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:34.725309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:34.732179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:34.732726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:34.733170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:34.733936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:34.734431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:34.734866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:34.770033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ecf1061-8da5-44db-bfbf-757e992365d4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051970 Sep 19 16:20:34.770682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 536/999] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:35.739827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:35.740577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:35.747751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:35.748235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:35.748670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:35.749372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:35.749823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:35.750245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:35.819135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:35.828347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cc22c754-2849-45e6-852e-7097e8523ca7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092968 Sep 19 16:20:35.828943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 537/1000] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:35 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:35.833829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:35.836028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:35.842421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:35.842915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:35.843349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:35.844057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:35.844510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:35.844930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:35.879609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f093cfe0-042f-4f27-a9db-804064e22146 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049549 Sep 19 16:20:35.880537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 538/1001] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:36.854151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:36.855080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:36.863021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:36.863510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:36.863947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:36.864662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:36.865117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:36.865546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:36.939162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:36.948449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-56c7ca3b-5a5f-45c0-a926-0ecde54b7490 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100175 Sep 19 16:20:36.949145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 539/1002] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:36 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:36.962554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:36.966909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:36.980572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:36.981070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:36.981489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:36.982211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:36.982691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:36.983121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:37.024095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-99af3130-d799-4514-a09c-98286b7bb0ae tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.073508 Sep 19 16:20:37.024749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 540/1003] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 75 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:37.986282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:37.987036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:37.995282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:37.996136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:37.996906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:37.998144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:37.998980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:37.999680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:38.071538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:38.084859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6fbcbe92-c2ea-4c59-8aeb-3dbfe18d3d6b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102169 Sep 19 16:20:38.085741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 541/1004] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:37 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:38.093619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:38.096277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:38.104272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:38.104912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:38.105444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:38.106262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:38.106927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:38.107671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:38.142873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aa3c5be1-a0f5-44c2-a17b-d67a62a13407 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055526 Sep 19 16:20:38.143548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 542/1005] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:39.113201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:39.114014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:39.121447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:39.121936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:39.122343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:39.123094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:39.123545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:39.123971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:39.201284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:39.209747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb9a6d08-29d6-429a-8452-547f2481d400 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100436 Sep 19 16:20:39.210338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 543/1006] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:39 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:39.215736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:39.218005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:39.224633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:39.225115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:39.225548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:39.226253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:39.226736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:39.227153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:39.260339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7b466b34-5116-4af6-8bec-377bf0ec183c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048855 Sep 19 16:20:39.261480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 544/1007] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:40.239443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:40.240407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:40.251204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:40.251698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:40.252138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:40.252863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:40.253318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:40.253743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:40.328299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:40.331752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:40.337417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d21f5f47-9f73-4c78-9629-f1de294318a5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.101973 Sep 19 16:20:40.338642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 545/1008] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:40 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:40.343691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:40.345916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:40.352528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:40.353049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:40.353497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:40.354239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:40.354808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:40.355346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:40.390276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-341a17c4-8ba2-40fb-9e61-0e2db19a5922 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050571 Sep 19 16:20:40.390895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 546/1009] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:41.375973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:41.377300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:41.383402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:41.383895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:41.384351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:41.385064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:41.385523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:41.385962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:41.464187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:41.474215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1d75df07-4d16-4efc-960c-99534bed9a0c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102436 Sep 19 16:20:41.474858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 547/1010] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:41 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:41.480402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:41.482610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:41.489208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:41.489743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:41.490239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:41.491072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:41.491573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:41.492033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:41.531201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d977891b-4d04-4350-9c8e-be5b097955a8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055231 Sep 19 16:20:41.531795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 548/1011] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:42.501101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:42.501896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:42.508936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:42.509422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:42.509863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:42.510651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:42.511078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:42.511504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:42.581975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:42.591328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e80a6ffa-4b6c-4bf6-9a6f-80fbddc24315 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093970 Sep 19 16:20:42.591741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 549/1012] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:42 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:42.596638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:42.598915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:42.605680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:42.606173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:42.606653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:42.607372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:42.607829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:42.608263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:42.643140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a6bf0a4b-313d-403c-b1ab-be88b9d70286 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050305 Sep 19 16:20:42.643786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 550/1013] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:42 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:43.618177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:43.619318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:43.626338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:43.626869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:43.627312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:43.628031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:43.628492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:43.628921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:43.720167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:43.728569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0424e36d-2e66-4812-b262-464b61505112 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.114899 Sep 19 16:20:43.729174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 551/1014] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:43 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:43.734850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:43.737108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:43.743810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:43.744894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:43.745422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:43.746203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:43.746695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:43.747126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:43.793871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d7dc460c-74e6-4275-9b58-ed99b89c52f2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061558 Sep 19 16:20:43.794618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 552/1015] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:44.755342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:44.756089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:44.762981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:44.763471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:44.763908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:44.764624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:44.765074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:44.765500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:44.832685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:44.841743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-875ed8d2-c742-4e9c-9c8b-20c2570a9446 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090143 Sep 19 16:20:44.842241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 553/1016] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:44 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:44.847301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:44.849486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:44.857032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:44.857515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:44.857989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:44.858718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:44.859196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:44.859620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:44.892083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c40cc189-7f5b-460d-a115-0c8bbb9ec2a7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048730 Sep 19 16:20:44.892654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 554/1017] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:45.865484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:45.866274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:45.873705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:45.874202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:45.874677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:45.875384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:45.875831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:45.876270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:45.944796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:45.953270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-09e08bd0-521b-4b06-bb54-acf6ea3eaa42 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091810 Sep 19 16:20:45.953950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 555/1018] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:45 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:45.963743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:45.967835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:45.976574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:45.977140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:45.977718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:45.979347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:45.979954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:45.980494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:46.014356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7dbdc090-b278-4e18-9a52-bc3fef2673a4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058900 Sep 19 16:20:46.014990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 556/1019] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:46.986681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:46.987429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:46.994556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:46.995042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:46.995473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:46.996188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:46.996634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:46.997055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:47.065272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:47.074907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8ae5fd0b-101d-4a53-bd89-fdf87873c497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092116 Sep 19 16:20:47.075668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 557/1020] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:46 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:47.081526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:47.084237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:47.092218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:47.092700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:47.093166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:47.093883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:47.094330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:47.094807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:47.130663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-353112b6-22d8-4215-9509-63f7c3fc10b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053669 Sep 19 16:20:47.131269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 558/1021] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:48.100707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:48.101489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:48.109350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:48.109834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:48.110273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:48.111039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:48.111492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:48.111926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:48.183242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:48.193896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-690429f9-d30e-4358-a346-1592e8d1a924 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.097656 Sep 19 16:20:48.194529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 559/1022] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:48 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:48.199437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:48.201658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:48.208837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:48.209327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:48.209762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:48.210557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:48.211014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:48.211435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:48.244943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e988d7f6-84b7-40e8-8507-49de26683a27 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049314 Sep 19 16:20:48.245547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 560/1023] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:49.220281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:49.221035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:49.227544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:49.228271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:49.229537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:49.230257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:49.230732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:49.231161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:49.296551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:49.304890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e091e71b-4d05-480c-88aa-5f0c5c406692 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088751 Sep 19 16:20:49.305478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 561/1024] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:49 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:49.310979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:49.313319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:49.320331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:49.320819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:49.321262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:49.322707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:49.323261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:49.323693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:49.344323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:49.358788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9781acd6-e90c-4121-9c9d-7fb3a6c91ee7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052208 Sep 19 16:20:49.359355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 562/1025] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:50.329037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:50.329801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:50.336527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:50.337010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:50.337455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:50.338839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:50.339325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:50.339751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:50.406116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:50.414117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d2d0670e-6f48-4c9b-a68c-f444932b4c9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.089150 Sep 19 16:20:50.414735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 563/1026] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:50 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:50.419775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:50.422180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:50.428361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:50.428838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:50.429279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:50.429975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:50.430447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:50.430902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:50.465108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2de453a4-a7bc-4128-9db3-f602ca936e3e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049282 Sep 19 16:20:50.465700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 564/1027] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:51.437583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:51.438272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:51.445135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:51.445615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:51.446046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:51.446780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:51.447226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:51.447650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:51.516073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:51.531302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-96d7d10b-d6b4-4726-8753-5498393016ac tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.096894 Sep 19 16:20:51.532010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 565/1028] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:51 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:51.540802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:51.548672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:51.555376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:51.555863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:51.556294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:51.557044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:51.557495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:51.557921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:51.591856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-977edce8-f3ea-4c9a-ab4f-b7e86fc89935 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058493 Sep 19 16:20:51.592450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 566/1029] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:52.555920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:52.556727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:52.563883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:52.564371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:52.564811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:52.565556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:52.566011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:52.566477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:52.636195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:52.647209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c521a64-aecb-426c-afa6-8f33910d3a80 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095120 Sep 19 16:20:52.648015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 567/1030] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:52 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:52.657782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:52.661284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:52.671876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:52.672753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:52.673538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:52.674884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:52.675718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:52.676505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:52.714048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8f893987-ae44-4e41-bfc4-ec62092d480c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064575 Sep 19 16:20:52.715029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 568/1031] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:53.697696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:53.698904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:53.710953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:53.711816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:53.713121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:53.714511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:53.715383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:53.716194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:53.790443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:53.800546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-28355244-f551-4690-a7da-a8cee47bd266 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.110418 Sep 19 16:20:53.801288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 569/1032] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:20:53.807060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:53.809587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:53.816936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:53.817557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:53.818069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:53.818893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:53.819442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:53.820095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:53.856558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fc443a55-ebf6-42e6-8276-ea8cc57e4b85 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054033 Sep 19 16:20:53.857157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 570/1033] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:54.826465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:54.827130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:54.833550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:54.834022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:54.834482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:54.835175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:54.835615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:54.836036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:54.904620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:54.912930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-00b6591e-e544-49b8-b4b3-31d96b0e6016 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090271 Sep 19 16:20:54.913869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 571/1034] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:54 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:54.919111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:54.921357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:54.928484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:54.929009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:54.929400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:54.930432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:54.930594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:54.931007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:54.964820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f2ae0db5-f25a-4246-aba7-04c5a2f91177 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050150 Sep 19 16:20:54.965418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 572/1035] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:55.938398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:55.939482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:55.946555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:55.947192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:55.947686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:55.948434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:55.948958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:55.949427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:56.036779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:56.048309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e78427cb-1b0a-4c71-9368-9ce29b5a835e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.113795 Sep 19 16:20:56.048927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 573/1036] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:55 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:56.053849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:56.056063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:56.064518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:56.065005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:56.065451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:56.066193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:56.066677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:56.067102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:56.107615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d777c8f0-e9b8-43b4-b405-184d2bb3cd65 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057574 Sep 19 16:20:56.108218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 574/1037] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:57.073614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:57.074414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:57.081196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:57.081731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:57.082340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:57.083202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:57.083751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:57.084385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:57.160018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:57.169613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-53092b00-45c5-4501-bfc5-daf036c47a12 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100029 Sep 19 16:20:57.170208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 575/1038] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:57 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:57.175517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:57.177757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:57.184889 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:57.185366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:57.185798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:57.186571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:57.187018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:57.187441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:57.220934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8f4184e1-91f4-4510-93cd-8c48406c0746 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049573 Sep 19 16:20:57.221543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 576/1039] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:58.194171 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:58.194901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:58.202726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:58.203262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:58.203737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:58.204566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:58.205055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:58.205553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:58.275727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:58.284751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0b97791a-4fba-4c7f-aa5a-3f59922757c1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094610 Sep 19 16:20:58.285669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 577/1040] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:58 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:58.290550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:58.292893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:58.299991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:58.300775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:58.301383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:58.302342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:58.303018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:58.303833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:58.341994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-47df5950-7fea-4a61-92f6-f05e4781af50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055485 Sep 19 16:20:58.342642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 578/1041] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:20:59.309476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:59.310209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:59.316809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:59.317348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:59.317818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:59.318951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:59.319492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:59.319956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:59.325321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:20:59.396390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:20:59.405961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f67548b3-7725-4784-98b3-287ab491d880 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100514 Sep 19 16:20:59.406605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 579/1042] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:20:59 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:20:59.411961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:20:59.416667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:20:59.424466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:59.424958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:59.428012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:59.428760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:20:59.429227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:20:59.429661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:20:59.466598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-182d0adc-011c-4ee8-b802-a93f3906e260 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058804 Sep 19 16:20:59.467288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 580/1043] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:20:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:00.434722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:00.435439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:00.437983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:00.445739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:00.446240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:00.446710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:00.447978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:00.448440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:00.449142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:00.532848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:00.545487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f13ee60c-9533-4bca-89f3-2cb01d5f20d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.114970 Sep 19 16:21:00.546454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 581/1044] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:00 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:00.552300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:00.554608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:00.561820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:00.562312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:00.562792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:00.563518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:00.563975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:00.564405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:00.606465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9c74859d-9fa7-4c14-a8ba-2146de797b3d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058347 Sep 19 16:21:00.607230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 582/1045] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:01.437145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:01.438186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:01.445802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.446286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.446750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.447502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.447908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.448324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.476914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:21:01.571480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a9dd77ab-1ba5-4a34-b3c0-9123fe3b0d87 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 204 len: 0 microversion: 2.1 time: 0.137792 Sep 19 16:21:01.572686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 583/1046] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:21:01 2022] DELETE /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:21:01.583446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:01.584342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:01.591215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.591822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.592301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.593048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.593613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.594101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.768533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:01.777299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-339aa223-8e1c-4fcb-81ef-f5db6a2b2cfb tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1682 microversion: 2.1 time: 0.198966 Sep 19 16:21:01.778694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 584/1047] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:01 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1682 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:01.788076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:01.789247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:01.801153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.802048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.802904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.804199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.805474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.806647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.878949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:01.887340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-763809e7-c818-4da1-93e9-48911053506e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.107053 Sep 19 16:21:01.887949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 585/1048] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:01 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 109 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:01.893130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:01.895440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:01.903174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.903787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.904288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.905835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:01.905835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:01.905835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:01.940452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-39c8416c-3f34-4886-9d15-709a897cb278 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051388 Sep 19 16:21:01.941146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 586/1049] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:02.816392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:02.817431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:02.825205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:02.825837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:02.826393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:02.827235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:02.827785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:02.828497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:02.986859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:02.996122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-187714af-b226-44b2-b217-8a09b4dad033 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1682 microversion: 2.1 time: 0.183693 Sep 19 16:21:02.996721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 587/1050] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:02 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1682 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:03.002027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:03.002672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:03.009967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:03.010501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:03.010943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:03.011815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:03.012120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:03.012555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:03.085262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:03.095430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bc600e90-d678-4342-990b-9df4af308485 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.097628 Sep 19 16:21:03.096462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 588/1051] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:02 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:03.103674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:03.106243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:03.113880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:03.114686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:03.115288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:03.116157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:03.116788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:03.117501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:03.154549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8cc83a68-7571-4037-8bcd-1c407635ff53 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056819 Sep 19 16:21:03.155150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 589/1052] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:03.938279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a34c2a71-df97-4e4d-ab68-da770b22a036", "name": "network-vif-deleted", "tag": "5bdbe7af-21f9-4fe2-8859-99f053e3c9e2"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:03.957320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:03.957981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:03.958622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:03.979448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] Creating event network-vif-deleted:5bdbe7af-21f9-4fe2-8859-99f053e3c9e2 for instance a34c2a71-df97-4e4d-ab68-da770b22a036 on hv-cinder-85837 Sep 19 16:21:03.985686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-c6ec252d-79c6-4826-899b-d0820cb05636 req-84e51466-9714-4ddc-b066-a82a66cdfa3d service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055417 Sep 19 16:21:03.986714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 590/1053] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:21:03 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:04.032791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:04.034095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:04.059492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.060464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.061170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.062431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.063355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.064117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.153981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.155832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.156642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.157411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lazy-loading 'fault' on Instance uuid a34c2a71-df97-4e4d-ab68-da770b22a036 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:21:04.231755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:04.247641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9dec86ac-53d3-445b-9ec8-1b2c4462cf8c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 200 len: 1331 microversion: 2.1 time: 0.224731 Sep 19 16:21:04.252342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 591/1054] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:04 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 1331 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:04.262312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:04.263585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:04.276144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.277123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.278008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.279717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.280768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.281645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.388012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:04.405112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a8f0d63-56f3-4985-b0f8-780345fbe3a8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.150770 Sep 19 16:21:04.418526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 592/1055] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:04 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:04.418526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:04.419142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:04.432324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.433184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.434092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.435492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:04.436459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:04.437209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:04.502336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-654f8dca-8125-452e-b9dd-c24293d9d7ce tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.094125 Sep 19 16:21:04.511740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 593/1056] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:05.297698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:05.298908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:05.311174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.311986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.312695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.313504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.314040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.314849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.334709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 HTTP/1.1 Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:21:05.338471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:21:05.341088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'a34c2a71-df97-4e4d-ab68-da770b22a036'} Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:21:05.342576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:21:05.345430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:21:05.348854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:21:05.348854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fb933a5f-4838-4ec8-8d71-be718b5b123c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 404 len: 111 microversion: 2.1 time: 0.051129 Sep 19 16:21:05.348854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 594/1057] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:05 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 111 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:21:05.355926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-07c00e51-3a4d-4f0b-9692-dc17ebe8a223 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '.version_select at 0x7fadbf6a0700>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:05.380237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-07c00e51-3a4d-4f0b-9692-dc17ebe8a223 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2097141825" status: 202 len: 0 microversion: 2.1 time: 0.031124 Sep 19 16:21:05.380934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 595/1058] 10.222.0.57 () {58 vars in 1221 bytes} [Mon Sep 19 16:21:05 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2097141825 => generated 0 bytes in 33 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:21:05.443116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:05.444161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:05.459315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.460225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.461139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.463448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.464428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.464963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.473476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:05.596206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:05.608949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7a98335f-43ea-4668-8bf8-e2d727351718 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.173601 Sep 19 16:21:05.609976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 596/1059] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:05 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:05.615019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:05.620867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:05.624246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.624879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.625216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.625964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:05.626475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:05.626929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:05.675591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-43dcff92-c69c-4a66-9db2-b78d5ef8c4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064573 Sep 19 16:21:05.683394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 597/1060] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:06.236234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-de3c3408-c8a0-4d3e-91f6-71b715155d33 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: .version_select at 0x7fadbf8444c0>, body: {"keypair": {"name": "tempest-keypair-27961422", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBM44GEbmrJUSST/gKcqEwyeupbWPdd8vQIUlX41AILM2sirJ7BSa9/rqzutn1ipG/tt70y2uJxNE+DhlQkS//Zb3EVvxsIiDXsRkQU4DDZFGKAazqbTQnEAIHa1PKA7j9A=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:06.253421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-de3c3408-c8a0-4d3e-91f6-71b715155d33 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:21:06.257863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-de3c3408-c8a0-4d3e-91f6-71b715155d33 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:21:06.286067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-de3c3408-c8a0-4d3e-91f6-71b715155d33 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.054070 Sep 19 16:21:06.286738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 598/1061] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:21:06 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:06.641678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:06.642646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:06.653999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:06.654886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:06.655575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:06.656453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:06.657087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:06.657734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:06.768831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:06.786182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ea101b45-9940-40c6-a93a-316ea034f21c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.148963 Sep 19 16:21:06.801621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 599/1062] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:06 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:06.815312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:06.820373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:06.842443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:06.843913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:06.845099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:06.846822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:06.848046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:06.849104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:06.902831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-807bdace-f13f-4a9e-abcb-bce1bb92dc9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.098910 Sep 19 16:21:06.921619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 600/1063] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:07.396133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:21:07.494313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:07.495994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:07.509366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:07.510299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:07.511409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:07.512925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:07.514468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:07.515262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:14.834954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ffad7fe6-56bf-4331-b19e-d2cfc40e12c6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 202 len: 0 microversion: 2.1 time: 50.631545 Sep 19 16:21:14.835986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 464/1064] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:20:24 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 50634 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:21:14.845539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:14.846582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:14.858288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:14.859150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:14.860174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:14.861398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:14.862160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:14.863509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:14.896659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:21:14.965631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:14.974220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b8f00c22-935d-43a4-921b-2d5929ad0892 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.135961 Sep 19 16:21:14.974893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 465/1065] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:14 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:14.979873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:14.982146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:14.988723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:14.989210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:14.989647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:14.990364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:14.990849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:14.991276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:15.024026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2afc87ef-75e4-411f-ada7-0c3682d5e2b8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047855 Sep 19 16:21:15.025055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 466/1066] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:15.030325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-2051734741", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "145a1b1d-9f41-428a-98d1-15b34e55a4ad"}], "security_groups": [{"name": "tempest-securitygroup--1927393958"}], "key_name": "tempest-keypair-27961422", "user_data": "}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:15.160694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] validate_networks() for [('145a1b1d-9f41-428a-98d1-15b34e55a4ad', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:21:15.813581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:21:15.814014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:21:15.814523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:21:15.814929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:21:15.815379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:21:15.826499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:15.827327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:15.827774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:15.829021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:15.829402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:15.830192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:15.854545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:21:15.861589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Getting quotas for user fd36089b3c2141218660ab2458376798 and project a1842253d6634f1f8b93bfa5be2ddd5f. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:21:15.875287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:21:15.877926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:21:15.930228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0a96b25e-ddb8-4f34-828a-14092251c95e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 0.904055 Sep 19 16:21:15.933422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 467/1067] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:21:15 2022] POST /compute/v2.1/servers => generated 402 bytes in 906 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:21:15.937354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:15.940025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:15.947354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:15.947956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:15.948274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:15.949005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:15.949452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:15.949878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:15.986104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-42ca85f2-e211-4244-aa29-56199b4ad740 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053339 Sep 19 16:21:15.986908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 468/1068] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:15 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:15.992201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d29d65f2-dcf4-4ac2-8b1d-7beb895979d0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:15.992761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d29d65f2-dcf4-4ac2-8b1d-7beb895979d0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:16.053810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d29d65f2-dcf4-4ac2-8b1d-7beb895979d0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:16.063132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d29d65f2-dcf4-4ac2-8b1d-7beb895979d0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1240 microversion: 2.1 time: 0.074601 Sep 19 16:21:16.064009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 469/1069] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:15 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1240 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:16.069915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:16.070879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:16.079430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:16.080248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:16.080817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:16.082001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:16.082829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:16.083603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:16.154958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:16.165911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e233542-10c2-458f-819d-c02e2a08f0cf tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100492 Sep 19 16:21:16.166787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 470/1070] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:16 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:16.173376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:16.176674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:16.190206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:16.191210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:16.191930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:16.193105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:16.194013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:16.194663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:16.248065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4def4052-7053-4f3f-8daa-84503dca7026 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.079586 Sep 19 16:21:16.249135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 471/1071] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 82 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:16.267476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:16.999803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:17.002025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:17.009525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.010013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.010474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.011180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.011629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.012050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.045629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9ffc06f9-8148-4354-b3e7-a0efa5b9b8b9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049772 Sep 19 16:21:17.046216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 472/1072] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:17.087138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:17.087671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:17.094006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.094527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.094962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.095668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.096177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.096533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.163627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:17.173748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-65a3ed5c-0792-4f04-9efb-7fddaad5ebde tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1292 microversion: 2.1 time: 0.090435 Sep 19 16:21:17.175378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 473/1073] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:17 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1292 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:17.197354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:17.198542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:17.209868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.210736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.211500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.213156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.213640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.214071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.471074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:17.487375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7d3a8b67-2f41-4268-9be8-1023a7de99c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.298145 Sep 19 16:21:17.488233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 474/1074] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:17 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:17.498833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:17.501920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=58,nova_api:UPDATE=1,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:17.506099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:17.514300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.514826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.515269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.515982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:17.516658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:17.518101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:17.558354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cd6e383e-dcb9-41f6-b858-b6cf09a793cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.068491 Sep 19 16:21:17.559157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 475/1075] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:17 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:18.061557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:18.064480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:18.074537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.075033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.075469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.076211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.078460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.078906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.116005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-04bf2ba2-85de-4678-9cbb-981ac1224476 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.060044 Sep 19 16:21:18.116601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 476/1076] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:18 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:18.199235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:18.199893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:18.206502 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.206962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.207399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.209747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.210216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.210678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.333922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:18.344041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-790ea6d5-ce9a-4231-abb6-8e5d416f1840 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1298 microversion: 2.1 time: 0.149286 Sep 19 16:21:18.344636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 477/1077] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:18 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1298 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:18.519868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:18.520449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:18.529680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.530245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.530768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.531553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.532024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.532635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.619159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:18.632767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-52c6c65f-9a5c-4c6a-8265-455325dff7ec tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.119905 Sep 19 16:21:18.633354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 478/1078] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:18 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:18.652361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:18.654754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:18.661737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.662233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.662934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.663654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:18.664111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:18.664571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:18.709506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1089a3e1-3fab-49c5-9486-7ef0b73941db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061104 Sep 19 16:21:18.710781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 479/1079] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:19.129624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:19.132040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:19.138416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.138897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.139331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.140031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.140484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.140901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.175130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-903986dc-5d03-47f0-bf2b-77ca1c101427 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049690 Sep 19 16:21:19.175729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 480/1080] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:19 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:19.373038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:19.374059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:19.392902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.393767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.394579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.396019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.396601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.397355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.645794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:19.656025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8c37dfc9-1efa-47ef-97c8-4d46705e502a tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.289416 Sep 19 16:21:19.657550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 481/1081] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:19 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:19.664337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cf7e835d-ed2f-4b40-a16b-7e12ebd67527", "tag": "b963629b-7dec-4fd4-9bfe-f02f8aece1c2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:19.677919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.678602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.679145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.684893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:19.697003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] Creating event network-changed:b963629b-7dec-4fd4-9bfe-f02f8aece1c2 for instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 on hv-cinder-85837 Sep 19 16:21:19.703482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-7d7f1aab-6c44-4146-a8cd-c3a2c67b30de req-a045a265-5fdb-4d4b-afea-5566b4023476 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044572 Sep 19 16:21:19.704857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 482/1082] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:21:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:19.715406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:19.716698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:19.729356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.730265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.731104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.732442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.733271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.734061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.839178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:19.849683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-583dedb1-a673-4e45-b3f1-426dc4f1c456 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.142329 Sep 19 16:21:19.850462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 483/1083] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:19 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 145 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:19.859693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:19.863908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:19.876231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.877128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.877928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.879264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:19.880120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:19.880562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:19.927134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c244d91b-9f5f-4c2c-bd84-ceb6c42591ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074926 Sep 19 16:21:19.928373 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 484/1084] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:20.188612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:20.190915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:20.198394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.198889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.199334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.200055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.200530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.200956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.237047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e13ae64f-f71c-47a2-aab6-f864cae0a3b8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052878 Sep 19 16:21:20.237652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 485/1085] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:20 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:20.696651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:20.697284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:20.704277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.704771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.705212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.705978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.706460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.706886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.856176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:20.865178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1080bf11-c7bf-4a77-bbed-6f51edcfac52 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.172569 Sep 19 16:21:20.865688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 486/1086] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:20 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:20.887404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:20.888022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:20.894419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.894903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.895335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.896052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:20.896498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:20.896916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:20.970870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:20.981983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ef09b1f-732c-4dce-a2dc-78407b50f373 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098149 Sep 19 16:21:20.983021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 487/1087] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:20 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:20.990106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:20.993838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:21.002539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.003203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.003931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:21.004835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.005388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.006073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:21.042553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-659e2729-ef4c-4985-9aea-36fa7176ab9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058006 Sep 19 16:21:21.043146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 488/1088] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:21.251111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:21.253639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:21.260730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.261220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.261654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:21.262417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.262878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.263299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:21.303830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f62c6009-3e29-4a71-8a45-c86f37b03e54 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057183 Sep 19 16:21:21.304598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 489/1089] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:21 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:21.889750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:21.890451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:21.897370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.897862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.898310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:21.899056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:21.899563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:21.899993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.053522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:22.067080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-30040790-df0a-46c8-852d-fb782c09f0e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.181017 Sep 19 16:21:22.067977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 490/1090] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:21 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:22.077215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:22.078309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:22.088644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.089520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.090338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.091668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.092465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.093217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.163383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:22.172003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7d051b11-02c7-4dc0-baca-f3a3818c91d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102271 Sep 19 16:21:22.172613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 491/1091] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:22 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:22.177495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:22.179799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:22.186748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.187244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.187694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.188416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.188873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.189297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.224192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a8ba0fc2-f0e5-47d7-97a3-c919bd4fa423 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050424 Sep 19 16:21:22.224951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 492/1092] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:22.323760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:22.327618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:22.339333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.340203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.340986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.342266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:22.343147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:22.343914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:22.380130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2336113-e8bd-4213-9e81-7f1c444ef360 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064714 Sep 19 16:21:22.380896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 493/1093] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:22 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:23.091604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:23.092305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:23.100147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.100774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.101219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.102042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.103080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.103553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.256495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:23.270329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dd366a65-0c6f-4cff-9a0c-d965c478be08 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.182279 Sep 19 16:21:23.271408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 494/1094] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:23 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:23.280216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:23.281316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:23.292747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.293618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.294416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.295683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.296511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.297536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.389009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:23.398430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6a3ba0d3-fdd3-4c8b-aa65-d9dab8c6372f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.125495 Sep 19 16:21:23.399028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 495/1095] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:23 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:23.403919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:23.406124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:23.412801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.413298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.413742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.414493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.415512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.415948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.449625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c8e256a7-5382-41f1-9bb2-2719acbf4ea7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049465 Sep 19 16:21:23.450269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 496/1096] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:23.455125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:23.457340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:23.464315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.464811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.465257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.465979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:23.466505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:23.466937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:23.500593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c1af128f-4efc-4a7a-a053-40f6afba00f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049248 Sep 19 16:21:23.501183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 497/1097] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:23 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:24.300885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:24.302052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:24.313984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.314882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.315667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.316972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.317812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.319052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.487520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:24.501264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0c8640a1-d414-4089-982a-fb304a38f274 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.207618 Sep 19 16:21:24.502181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 498/1098] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:24 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:24.511789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:24.511789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:24.522109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.522990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.523766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.525532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.526389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.527170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.565846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:24.625332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:24.634052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-188ce260-75fc-4237-8fc3-033a8c56ba75 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.129856 Sep 19 16:21:24.634682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 499/1099] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:24 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:24.639617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:24.641806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:24.648770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.649824 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.650260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.651012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.651457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.651877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.685188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b5321182-1ac4-4f62-86e5-fd6570d0ca2e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049402 Sep 19 16:21:24.685772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 500/1100] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:24 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:24.690793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:24.693627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:24.700412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.700907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.701344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.702048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:24.702812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:24.703242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:24.736721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-87ee9b7b-6b8c-4185-8436-a8a18359b1ca tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049866 Sep 19 16:21:24.737382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 501/1101] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:24 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:24.904817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=30 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:25.525105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:25.526039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:25.532798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.533326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.533744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.534504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.534989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.535402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.741838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:25.753984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f32b0b07-fd16-4513-a413-8120b19db277 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.233090 Sep 19 16:21:25.755241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 502/1102] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:25 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:25.762669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:25.763779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:25.775969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.776787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.777578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.778679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.779275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.779798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.840576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:25.886292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:25.895720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a578d601-3336-460a-96ec-0324bc5e9a69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.139547 Sep 19 16:21:25.896328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 503/1103] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:25 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:25.901459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:25.903815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:25.910711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.911245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.911645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.912404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.912867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.913295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.954072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-971958b4-8615-462e-b823-253ccbad3722 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056595 Sep 19 16:21:25.954713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 504/1104] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:25.959778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:25.962058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:25.965299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=97,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:25.969827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.970325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.970809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:25.971540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:25.972005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:25.972441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:26.011147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7271805a-2df5-40c8-a74d-7d3473c11219 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055119 Sep 19 16:21:26.012096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 505/1105] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:25 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:26.041496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf7e835d-ed2f-4b40-a16b-7e12ebd67527", "name": "network-vif-plugged", "status": "completed", "tag": "b963629b-7dec-4fd4-9bfe-f02f8aece1c2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:26.052240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:26.052679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:26.053948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:26.064916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=8 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:26.067540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] Creating event network-vif-plugged:b963629b-7dec-4fd4-9bfe-f02f8aece1c2 for instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 on hv-cinder-85837 Sep 19 16:21:26.073833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-02ced864-661d-4025-b668-fb13f4b2b2da req-27605772-c29f-487d-92f7-7db97fdd6ce0 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.038091 Sep 19 16:21:26.074513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 506/1106] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:21:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:26.789623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:26.790350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:26.797283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:26.797809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:26.798240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:26.798972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:26.799420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:26.799864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:26.946519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:26.958309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b79bd713-8b45-40db-aa1b-f56cd8d00d54 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1366 microversion: 2.1 time: 0.173450 Sep 19 16:21:26.959053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 507/1107] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:26 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1366 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:26.966617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:26.968178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:26.977060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:26.977649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:26.978182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:26.979113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:26.979619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:26.980136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:27.071968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:27.080094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5f477fbf-6743-452e-a606-18ae2d96b687 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.119986 Sep 19 16:21:27.080676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 508/1108] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:26 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:27.085474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:27.088301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:27.094737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:27.095229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:27.095660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:27.096395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:27.096873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:27.097413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:27.132606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b227c1e-eb20-4d4c-928a-6683d69b3382 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050601 Sep 19 16:21:27.134009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 509/1109] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:27.139591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:27.141765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:27.148193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:27.148725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:27.149297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:27.150082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:27.150660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:27.151125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:27.186338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f61153d6-f6e2-4499-b08e-9e6019fbe220 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050843 Sep 19 16:21:27.187027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 510/1110] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:27 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:27.990985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:27.993833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:27.999190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:27.999682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.000120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.000836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.001291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.001719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.153365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'fd36089b3c2141218660ab2458376798', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'a1842253d6634f1f8b93bfa5be2ddd5f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:28.161498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-67400d48-651d-40a7-ac2a-b2416e0bb0e2 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 200 len: 1556 microversion: 2.1 time: 0.174460 Sep 19 16:21:28.162081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 511/1111] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:27 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 1556 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:28.167544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:28.168162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:28.174796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.175289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.175745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.176434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.176951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.177379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.242210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:28.250262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a1a33033-c79a-46df-a50f-00b692ff5b71 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.087013 Sep 19 16:21:28.250886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 512/1112] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:28 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:28.255784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:28.257984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:28.264649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.265141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.265583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.266297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.266785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.267216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.299983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8bcd6ab7-b8cc-4ca9-827a-078b8895204d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048003 Sep 19 16:21:28.300834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 513/1113] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:28.305979 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:28.306501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:28.312619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.313102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.313538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.314242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.314722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.315154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.395812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cf84ca83-6f2d-4e37-9def-5d9e2aa4c638 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.094092 Sep 19 16:21:28.396436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 514/1114] 10.222.0.57 () {58 vars in 1275 bytes} [Mon Sep 19 16:21:28 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527/os-interface => generated 285 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:28.401642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:28.404033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:28.410408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.410920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.411471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.412205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:28.412665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:28.413091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:28.446704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ff4d32a7-cbd2-4e53-aeba-a307918bf76b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049064 Sep 19 16:21:28.447479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 515/1115] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:28 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:29.277962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:29.279153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:29.291522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.292411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.293208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.294497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.295354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.296133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.392418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:29.408410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-07196f0f-cb92-475a-ab5f-9ddce9f92fa6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.138303 Sep 19 16:21:29.409308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 516/1116] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:29 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:29.418816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:29.423287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:29.434532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.435432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.436220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.437557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.438439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.439044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.443821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:29.474772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5aeb8c4a-6fc2-4f97-9e6d-a1eaff099697 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063875 Sep 19 16:21:29.475257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 517/1117] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:29.480048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:29.482197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:29.488433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.488917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.489350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.490049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.490520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.490947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.523993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1b86fa03-8124-4a4d-b42c-b941a232e913 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047663 Sep 19 16:21:29.524576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 518/1118] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:29 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:29.809326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cf7e835d-ed2f-4b40-a16b-7e12ebd67527", "tag": "b963629b-7dec-4fd4-9bfe-f02f8aece1c2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:21:29.819654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:29.820165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:29.820603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:29.835121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] Creating event network-changed:b963629b-7dec-4fd4-9bfe-f02f8aece1c2 for instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 on hv-cinder-85837 Sep 19 16:21:29.840953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-bca7c5d9-9e48-4e1a-acc3-88b1f56d3f49 req-22440c12-2516-484d-885f-f1d469cb0648 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036350 Sep 19 16:21:29.841563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 519/1119] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:21:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:30.453667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:30.456465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:30.472921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.473750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.474509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.475290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.475797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.476268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.575272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:30.590760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-18b321c9-7b7f-4195-9f6a-9ecf94573e88 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.147094 Sep 19 16:21:30.592303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 520/1120] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:30 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:30.598997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:30.601820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:30.611173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.612024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.612776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.614044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.614897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.615726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.675998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5431be59-2d72-4bfb-98da-ffcd0559fe92 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.081647 Sep 19 16:21:30.677047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 521/1121] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:30.686503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:30.690503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:30.701896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.702821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.703615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.704934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:30.705817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:30.706624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:30.766253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3e3ac2a5-5642-4709-8cbe-be717d4ca4e2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087248 Sep 19 16:21:30.767366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 522/1122] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:30 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:31.614877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:31.615665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:31.622176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.622693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.623124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.623827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.624270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.624689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.699454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:31.707808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b3f82263-07eb-4054-90ec-10974d824ef6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.096733 Sep 19 16:21:31.708737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 523/1123] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:31 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:31.714352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:31.716633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:31.723329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.723827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.724267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.724988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.725507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.725990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.762741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fc5ae4b7-f92b-4370-b30b-7674427ddadc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053185 Sep 19 16:21:31.763331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 524/1124] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:31.783291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:31.787026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:31.798404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.799301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.800096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.801369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:31.802208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:31.803056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:31.863897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-828297d6-d3ba-4abb-87f6-028b89022b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.087814 Sep 19 16:21:31.865035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 525/1125] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:31 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:32.737120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:32.737903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:32.747437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.748109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.748772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.749594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.750186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.750759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.854258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:32.868298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2286d19e-321b-4f3c-bdad-25220b30a4b0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.136805 Sep 19 16:21:32.869179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 526/1126] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:32 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:32.878742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:32.882682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:32.893903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.894915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.895602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.896916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.897758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.898551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.933297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b86ca482-c57c-497f-82ad-ce207ee2bccc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062773 Sep 19 16:21:32.934269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 527/1127] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:32.939430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:32.941719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:32.947790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.948264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.948702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.949392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:32.949841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:32.950260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:32.984791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-494e4dfe-350b-42f8-9e82-e726b72a205c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049083 Sep 19 16:21:32.985442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 528/1128] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:32 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:33.907646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:33.908425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:33.915076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:33.915562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:33.915989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:33.916689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:33.917134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:33.917553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:33.986965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:33.995864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5a0d75a2-11ec-4342-b8f7-177bf0625846 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092093 Sep 19 16:21:33.996440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 529/1129] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:33 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:34.001247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:34.003446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:34.010516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:34.011005 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:34.011440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:34.012147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:34.012593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:34.013010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:34.046287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-13537b55-20f9-442a-80fa-d197bbba0bd6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048530 Sep 19 16:21:34.047131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 530/1130] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:34.055853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:34.058165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:34.064480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:34.064955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:34.065384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:34.066083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:34.066573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:34.066984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:34.100391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-eef6d147-ba04-4d46-bec3-d668ddf41406 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051789 Sep 19 16:21:34.101004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 531/1131] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:34 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:35.021551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:35.022519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:35.029694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.030180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.030650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.031368 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.031830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.032260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.104619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:35.115604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9bfdd59d-a560-4af8-ad97-f04f1a3182c0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098085 Sep 19 16:21:35.116198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 532/1132] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:35 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:35.121102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:35.123326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:35.129752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.130241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.130716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.131431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.131892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.132321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.166032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d235517f-6308-4ed9-9833-62788aa72b6d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048738 Sep 19 16:21:35.166734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 533/1133] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:35.171606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:35.174107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:35.180654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.181140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.181575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.182292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:35.182779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:35.183205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:35.202579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:35.219081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-00ac9a01-ff0a-402e-9d25-3854906370a9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051249 Sep 19 16:21:35.219670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 534/1134] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:35 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:36.140706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:36.141380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:36.148532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.149013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.149450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.150163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.150668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.151097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.226714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:36.236061 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-912d5223-27da-426a-b96a-7bf17247f875 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099038 Sep 19 16:21:36.236802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 535/1135] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:36 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:36.241705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:36.243986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:36.250649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.251143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.251595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.252302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.252757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.253183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.287778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6e53a745-1cb6-4472-993b-89fadf2d56c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049886 Sep 19 16:21:36.288390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 536/1136] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:36.293804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:36.296099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:36.302749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.303243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.303683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.304405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:36.304857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:36.305319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:36.338067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-68a115dd-f7b4-4e0a-9a7c-0ee037d1ac21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048600 Sep 19 16:21:36.338686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 537/1137] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:36 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:37.270746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:37.271479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:37.279697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.281806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.281806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.282331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.283126 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.283837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.371404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:37.380112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bd99327e-7beb-4181-8598-af8d3a2771d8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.113338 Sep 19 16:21:37.380772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 538/1138] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:37 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:37.386848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:37.389430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:37.396368 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.396854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.397290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.397999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.398483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.398941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.433204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cbb051d3-90da-4598-9daf-e03efc76a950 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051119 Sep 19 16:21:37.434148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 539/1139] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:37.441745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:37.444043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:37.450622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.451113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.451569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.452270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:37.452719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:37.453145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:37.488525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bec56eb3-7acd-4bc7-8aa4-c707afe7037a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053054 Sep 19 16:21:37.489154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 540/1140] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:37 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:38.404324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:38.405207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:38.412667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.413376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.413883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.415515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.416134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.416663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.488605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:38.500424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-72967d86-dfac-4cd2-8bde-49efe07419a0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099743 Sep 19 16:21:38.501456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 541/1141] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:38 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:38.510738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:38.514818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:38.526518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.527418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.528233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.529631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.530512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.531305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.568860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4cbfaebd-cb19-4f6a-9f01-411fd640478d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065820 Sep 19 16:21:38.569452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 542/1142] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:38.574421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:38.576600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:38.582761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.583240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.583672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.584369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:38.584812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:38.585232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:38.618470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-512859c9-9889-4909-828b-921683ebc432 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047881 Sep 19 16:21:38.619077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 543/1143] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:38 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:39.528441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:39.529457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:39.540893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.541774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.542590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.543821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.544626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.545356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.649233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:39.659322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-94af355d-93d8-4a5d-a314-982ebadd8f1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.138321 Sep 19 16:21:39.659912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 544/1144] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:39 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:39.665354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:39.667910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:39.675556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.676249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.676813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.677672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.678216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.678779 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.715076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cbe2e198-b23a-4241-9974-335d33e101e1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054026 Sep 19 16:21:39.715665 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 545/1145] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:39.720447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:39.722691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:39.729351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.729873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.730317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.731079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:39.731533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:39.731969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:39.765297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ace4ac39-f099-4e35-8d9d-edc8ee7fa3fa tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048502 Sep 19 16:21:39.766227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 546/1146] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:39 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:40.684193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:40.685028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:40.694841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.695373 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.695886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.696643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.697104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.697535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.769691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:40.779037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1f0ba86f-66e0-43cd-84e1-5bae6cd154d1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099243 Sep 19 16:21:40.779625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 547/1147] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:40 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:40.784770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:40.787020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:40.793994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.794517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.794961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.795675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.796132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.796589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.828864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ef488ae4-0a62-4ea2-8d48-1ccb23d29fc9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048153 Sep 19 16:21:40.829510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 548/1148] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:40.834911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:40.838083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:40.844601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.845068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.845510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.846224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:40.846709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:40.847136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:40.880127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-838d98da-b6b0-4793-8fdd-3941e49c9b67 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049479 Sep 19 16:21:40.880720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 549/1149] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:40 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:41.822933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:41.824032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:41.835836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:41.836722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:41.837653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:41.839018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:41.839914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:41.840687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:41.954984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:41.965877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dc9efdc1-bf7a-4a79-8a1c-5448ae12aad2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.150472 Sep 19 16:21:41.966987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 550/1150] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:41 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:41.974968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:41.977955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:41.988664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:41.989512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:41.990267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:41.993044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:41.993663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:41.994470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:42.001944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:42.052549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed127260-4f2a-4bc8-bf83-86a1b64437d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.083939 Sep 19 16:21:42.053355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 551/1151] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:42.059328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:42.061582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:42.067989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:42.068512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:42.068927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:42.069639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:42.070093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:42.070591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:42.107162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1c77f7e3-d354-4d22-bb04-ad87f68c77d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052619 Sep 19 16:21:42.115277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 552/1152] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:42 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:42.993079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:42.993758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:43.000590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.001075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.001509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.002214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.002710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.003135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.076239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:43.084688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e4ea3369-afa1-4f92-a380-382ad0362f82 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095567 Sep 19 16:21:43.085284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 553/1153] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:42 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:43.090856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:43.093122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:43.100442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.101039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.101316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.102559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.103023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.103542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.136717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3802cd12-0e4a-4610-8c11-f745d75cf257 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050300 Sep 19 16:21:43.137315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 554/1154] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:43.142196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:43.144524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:43.147638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:43.152592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.153083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.153521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.154234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:43.154728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:43.155154 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:43.188338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cd0bcef1-963d-4bbf-823b-61d02d78e9c5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049931 Sep 19 16:21:43.188957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 555/1155] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:43 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:44.113855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:44.114956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:44.121980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.122576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.123019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.123734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.124185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.124611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.193220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:44.203613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a0d76aa1-1d1b-47a0-9ce9-76d9baf23dad tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093805 Sep 19 16:21:44.204253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 556/1156] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:44 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:44.209500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:44.211827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:44.221012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.221888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.222702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.223927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.224744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.225586 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.267838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e5229e4-7012-4d57-b0a6-816e8d9ec1d2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.062406 Sep 19 16:21:44.268571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 557/1157] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:44.274601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:44.276950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:44.283545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.284079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.284562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.285283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:44.285745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:44.286176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:44.321634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2f9c10a7-d80a-4fcd-bb1d-25bebab902a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051892 Sep 19 16:21:44.322241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 558/1158] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:44 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:45.230214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:45.231071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:45.238606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.239151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.239648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.240407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.240907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.241375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.312846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:45.321473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-149f6f7d-d304-4f6e-9ba5-2844f2e7bf69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095841 Sep 19 16:21:45.322078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 559/1159] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:45 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:45.327258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:45.330212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:45.336831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.337312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.337789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.338506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.338963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.339392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.373041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1737f27a-2881-4870-b47d-57227b006fe1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049817 Sep 19 16:21:45.374003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 560/1160] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:45.382487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:45.384920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:45.391231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.391714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.392150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.392868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:45.393311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:45.393728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:45.428531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4735a016-966b-4033-a737-b93066de26a6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053061 Sep 19 16:21:45.429128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 561/1161] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:45 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:46.349718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:46.350519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:46.358549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.359069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.359529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.360380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.360970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.361472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.430429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:46.438974 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a8b02560-fe5d-4955-9df3-2964efeb6f41 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093333 Sep 19 16:21:46.439716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 562/1162] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:46 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:46.448514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:46.452299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:46.463177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.464074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.465004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.466352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.467314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.468018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.501738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-01ec0bd0-1219-4228-863e-ec2527b123f1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060600 Sep 19 16:21:46.502330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 563/1163] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:46 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:46.507130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:46.509363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:46.517098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.517975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.518810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.520077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:46.520550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:46.521288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:46.582967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d865ac32-35de-43f2-920d-7c413813e8d5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.079348 Sep 19 16:21:46.583560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 564/1164] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:46 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 81 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:47.477082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:47.477798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:47.485209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.485702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.486147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.486905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.487363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.487791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.557910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:47.566523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1dd8eb8f-84f9-45c0-bd4e-8e73d635b99b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094075 Sep 19 16:21:47.567117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 565/1165] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:47 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:47.572094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:47.574308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:47.581379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.581949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.582438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.583182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.583670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.584095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.622141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e0eb348-3554-4989-9dbc-8de6d27fe4e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053884 Sep 19 16:21:47.622812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 566/1166] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:47.627972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:47.630190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:47.637571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.638076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.638543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.639253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:47.639705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:47.640135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:47.667492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:47.673563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d564104b-8f78-4c25-bdfe-67bbfe34daf0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049651 Sep 19 16:21:47.674238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 567/1167] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:47 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:48.594596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:48.595348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:48.602674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.603161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.603598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.604317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.604768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.605194 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.675724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:48.686085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-77b28c4e-414b-492e-a724-1b58b38ba1eb tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095684 Sep 19 16:21:48.686778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 568/1168] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:48 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:48.692314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:48.694740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:48.702322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.702844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.703279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.703982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.704429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.704847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.748587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7c022aea-713b-4580-bd9a-23ea4becc493 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.060602 Sep 19 16:21:48.749171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 569/1169] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:48.753988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:48.756178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:48.763000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.763488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.763920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.764632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:48.765077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:48.765523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:48.798754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4eee6220-5599-436a-8c59-1d0b162466be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048410 Sep 19 16:21:48.799459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 570/1170] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:48 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:49.714776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:49.715876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:49.723471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.724101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.724632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.725576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.726130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.726691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.801516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:49.816709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f09b1176-7459-46ad-8967-da413ed79812 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.106308 Sep 19 16:21:49.817642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 571/1171] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:49 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:49.827276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:49.831465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:49.842552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.843418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.844208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.845495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.846330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.847112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.907728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0329002e-e041-4db7-977b-13adf2ef28e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.088094 Sep 19 16:21:49.908896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 572/1172] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:49.918035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:49.922517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:49.934746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.935622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.936427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.937709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:49.938613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:49.939406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:49.978333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fe70ec26-ac1a-4c62-b1d6-f58e30736327 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067866 Sep 19 16:21:49.978862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 573/1173] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:49 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:50.841372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:50.842029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:50.848927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:50.849406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:50.849842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:50.850614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:50.851060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:50.851481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:50.919254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:50.928616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e39bb56-0dcd-428e-aec2-c47b9c4ad325 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091120 Sep 19 16:21:50.929207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 574/1174] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:50 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:50.934257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:50.936530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:50.943034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:50.943530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:50.943966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:50.944692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:50.945627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:50.946068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:50.979155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4f0fb30d-a0bd-4d05-b797-18be17f7bf0f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048824 Sep 19 16:21:50.979748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 575/1175] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:50.990674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:50.992994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:50.999754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:51.000239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:51.000672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:51.001386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:51.001838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:51.002265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:51.035535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-eeb5ab65-3769-48c5-a7d2-bd2cc8d02cd3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048816 Sep 19 16:21:51.036141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 576/1176] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:50 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:21:51.956045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:51.957121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:51.964777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:51.965417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:51.965897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:51.966859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:51.967361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:51.967895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:52.037338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:52.045859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fdd3f7fa-38eb-4349-ab1b-6751efc4e89c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093993 Sep 19 16:21:52.046471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 577/1177] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:51 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:52.052437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:52.055162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:52.061727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:52.062425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:52.063018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:52.063849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:52.064399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:52.064923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:52.101980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e81d384a-3efd-4fca-a1e3-7598b4f917e9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053686 Sep 19 16:21:52.103039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 578/1178] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:52.109245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:52.111662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:52.119786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:52.120409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:52.120942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:52.121755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:52.122302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:52.122935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:52.155902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fc1ad09c-0ae8-44a6-8e35-f2381873973a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051589 Sep 19 16:21:52.156690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 579/1179] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:52 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:53.070717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:53.071490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:53.078125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.078666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.079111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.079835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.080292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.080719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.147435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:53.156033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-61368a33-4378-492d-997e-7b920d396aaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.089798 Sep 19 16:21:53.156634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 580/1180] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:53.161629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:53.164448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:53.170960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.171650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.172354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.173068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.173519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.173951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.208275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c445db81-57de-4720-992b-a3cbc2160c1b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050531 Sep 19 16:21:53.208879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 581/1181] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:53.214091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:53.216378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:53.222948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.223993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.224205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.225190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:53.225817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:53.227102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:53.265037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-16a53e8a-d88c-4788-becc-0254fe333ff5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055073 Sep 19 16:21:53.265686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 582/1182] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:53 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:54.183680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:54.184430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:54.191192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.191770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.192276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.193057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.193511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.193932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.261337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:54.269537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-59945f8f-3acb-4beb-87df-51d1903cd630 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090405 Sep 19 16:21:54.270122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 583/1183] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:54 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:54.275197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:54.277388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:54.286805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.287773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.288560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.289987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.290873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.291650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.310870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:21:54.328468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1074cb25-2f75-458c-bfb0-73c27773b798 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057132 Sep 19 16:21:54.329049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 584/1184] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:54.334279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:54.336817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:54.343049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.343535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.343967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.344664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:54.345108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:54.345531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:54.377711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c0ec817e-316f-4dc5-83df-60482006ebda tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047520 Sep 19 16:21:54.378621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 585/1185] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:54 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:55.300747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:55.301457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:55.315187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.315749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.316223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.316986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.317516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.318260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.387361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:55.395709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-18cabe77-4bd0-4b67-9cb5-1b3c3e6d21c5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099059 Sep 19 16:21:55.396312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 586/1186] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:55 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:21:55.402300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:55.404585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:55.411430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.411929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.412372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.413090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.413546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.414054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.447860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0135949c-7733-4848-99d9-6a550e72d136 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050421 Sep 19 16:21:55.448459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 587/1187] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:55.453320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:55.455540 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:55.462659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.463407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.463955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.464829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:55.465389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:55.465921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:55.499696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-409efb38-73dc-454d-8952-659149020515 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050135 Sep 19 16:21:55.500390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 588/1188] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:55 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:56.425001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:56.425874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:56.433354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.433840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.434275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.435012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.435463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.435886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.506262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:56.514814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-940e61c6-1327-415c-9b00-108a50313d60 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.097857 Sep 19 16:21:56.526335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 589/1189] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:56 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:56.526335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:56.526335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:56.530637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.531184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.531624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.532341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.532795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.533258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.579850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a16794aa-89d6-41ab-b2fd-822b919e7281 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.063321 Sep 19 16:21:56.580453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 590/1190] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:56.585292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:56.587993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:56.594592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.595078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.595520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.596251 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:56.596677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:56.597093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:56.630849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-53d32b87-f3a4-4d59-a51e-a71b7bacb2f9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049291 Sep 19 16:21:56.631458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 591/1191] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:56 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:57.541684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:57.542423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:57.549472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.549964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.550433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.551160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.551620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.552051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.618447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:57.626807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8fd9fa33-b6ca-4cbd-a993-c0876ba59ce1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.089458 Sep 19 16:21:57.627395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 592/1192] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:57 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:57.632190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:57.634715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:57.641171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.641662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.642105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.642851 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.643301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.643732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.676756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-799b7791-8e0a-477a-821b-e143a3920bac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048267 Sep 19 16:21:57.677468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 593/1193] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:57.686678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:57.689280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:57.696656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.697522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.697982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.698734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:57.699197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:57.699627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:57.734732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-508acc05-0fa2-4503-ba77-c42677be4e03 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.055798 Sep 19 16:21:57.735332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 594/1194] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:57 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:58.650811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:58.651527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:58.658131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.658652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.659085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.659800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.660257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.660684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.725373 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:58.735103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a972b1e9-f790-4e22-8e2c-cda6b883eb83 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088725 Sep 19 16:21:58.735856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 595/1195] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:58 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:58.742227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:58.744863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:58.752232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.752810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.753347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.754121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.754699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.755200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.794560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3a1f6c68-602e-4969-b6b9-bb61fe69a1cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057419 Sep 19 16:21:58.795212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 596/1196] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:58.801353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:58.803752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:58.811551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.812179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.812720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.813539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:58.814136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:58.814668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:58.853418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8930b90c-9dd2-4e8e-8f5d-565117d4323b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056982 Sep 19 16:21:58.854127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 597/1197] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:58 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:21:59.761123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:59.762144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:59.768897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.769391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.769832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.770668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.771124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.771552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.841625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:21:59.851524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c65804c3-23d0-42f1-9cbb-646033953d93 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093902 Sep 19 16:21:59.852563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 598/1198] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:21:59 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:21:59.858918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:59.861963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:59.870823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.871477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.872021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.872872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.873490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.873989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.908505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-821305dc-f4b4-4f90-9f01-16e868d90fd2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054601 Sep 19 16:21:59.909094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 599/1199] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:21:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:21:59.914054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:21:59.916334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:21:59.922961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.923449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.923885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.924611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:21:59.925067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:21:59.925499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:21:59.960143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e39765c5-3c84-403e-9485-4bc38efbe720 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049946 Sep 19 16:21:59.961396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 600/1200] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:21:59 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:00.881857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:00.883046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:00.895014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:00.895806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:00.896603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:00.897914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:00.898796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:00.899588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:00.908018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:22:00.975829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:00.984484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3d8ee90d-146e-4636-8f4b-754764bbf844 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.110433 Sep 19 16:22:00.985058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 601/1201] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:00 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:00.989761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:00.991998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:00.998893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:00.999407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:00.999844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:01.000788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:01.001507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:01.002217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:01.035981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-131a749d-64c0-415e-94f3-03288b1383a1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049845 Sep 19 16:22:01.036651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 602/1202] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:01.046049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:01.048636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:01.055121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:01.055741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:01.056291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:01.057109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:01.057767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:01.058515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:01.091309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9a371a74-5dfe-4143-be73-a8ec2c67d01d tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053230 Sep 19 16:22:01.091893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 603/1203] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:01 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:02.009195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:02.010044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:02.017227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.017820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.018308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.019137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.019647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.020212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.091324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:02.100242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7bc19c9b-aba8-4835-aec5-e62653c5fbff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095003 Sep 19 16:22:02.100850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 604/1204] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:02 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:02.106010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:02.108294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:02.111415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:22:02.116300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.116796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.117243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.117966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.118443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.118873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.153387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dc826c41-7cb1-4054-981b-16762d7c977c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051316 Sep 19 16:22:02.154176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 605/1205] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:02.159107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:02.161300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:02.168023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.168510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.168941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.169646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:02.170095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:02.170619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:02.203214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7ac1323b-2dc6-4b49-a8d9-91508c07d7e5 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047909 Sep 19 16:22:02.203812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 606/1206] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:02 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:03.127381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:03.128507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:03.134990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.135476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.135909 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.136614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.137062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.137479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.208646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:03.218954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-529cdd30-d3ea-461b-8ff7-f2408e3d3263 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095412 Sep 19 16:22:03.219625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 607/1207] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:03 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:03.224914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:03.227893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:03.234597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.235082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.235518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.236221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.236670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.237087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.275773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7b743f16-133a-400a-8139-807ee690ca35 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054917 Sep 19 16:22:03.276388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 608/1208] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:03.281769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:03.284093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:03.291272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.291769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.292214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.292944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:03.293401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:03.293835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:03.328818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d5ae25cb-2afd-486e-b3a2-42ec3458d9d8 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051276 Sep 19 16:22:03.329592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 609/1209] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:03 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:04.244991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:04.245693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:04.252440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.252929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.253362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.254068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.254567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.254998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.336585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:04.344754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bbfece44-015b-49eb-86a0-33f85e92b628 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.103606 Sep 19 16:22:04.345348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 610/1210] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:04 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:04.350439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:04.352672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:04.359492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.359985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.360429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.361163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.361618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.362052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.393864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a5b7205-0e49-41dc-916f-6e4286b03b5f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047411 Sep 19 16:22:04.394465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 611/1211] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:04.399874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:04.403064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:04.409250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.409742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.410187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.410935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:04.411392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:04.411834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:04.445256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-35ec9557-24dc-4fd7-9d41-5ed10426487c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049117 Sep 19 16:22:04.445873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 612/1212] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:04 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:05.369815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:05.370558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:05.377228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.377708 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.378135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.378872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.379317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.379733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.447311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:05.455846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-df3a65f1-436a-44f0-a948-410224d194c3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090142 Sep 19 16:22:05.456421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 613/1213] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:05 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:05.462050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:05.464433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:05.470719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.471220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.471648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.472355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.472810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.473238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.514138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5e08ccd1-3ea6-4009-b0bf-5a0ef116f27b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.056575 Sep 19 16:22:05.514764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 614/1214] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:05.519717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:05.521899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:05.528258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.528754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.529190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.529920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:05.530419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:05.530848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:05.563933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bd0143f8-3d18-4260-a766-a3edc248a2f7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048002 Sep 19 16:22:05.564525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 615/1215] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:05 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:06.279394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:06.280071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:06.286739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.287362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.288009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.289432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.289894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.290322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.317176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-66b3e6df-8db1-4a65-98e5-3bef6890d717 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527/ips" status: 200 len: 150 microversion: 2.1 time: 0.041984 Sep 19 16:22:06.317807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 616/1216] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:22:06 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:22:06.481892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:06.483047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:06.490820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.491359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.491840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.492567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.493021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.493444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.931234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:22:06.939624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a9800ac2-391d-476b-a01c-b4799e939d9f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.463563 Sep 19 16:22:06.940341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 617/1217] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:22:06 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 465 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:22:06.945539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:06.947881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:06.954956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.955476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.956028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.956906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:06.957456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:06.957960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:06.963039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:22:06.997374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7c35e42d-d200-4b4c-9916-61090596f210 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055917 Sep 19 16:22:06.997961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 618/1218] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:22:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:22:07.003090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:07.005524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:07.012774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:07.013263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:07.013732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:07.014564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:07.015083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:07.015544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:07.052792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c345f37c-6ba4-4c8d-98c4-6ae3168b27f4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053615 Sep 19 16:22:07.053387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 619/1219] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:22:06 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:22:07.549051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:22:07.549761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:22:07.557038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:07.557614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:07.558179 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:07.559145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:22:07.559793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:22:07.560353 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:22:07.761102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:22:07.761410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:22:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:22:17.554858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=30 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:22:17.584967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=14 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:07.575555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b7580eb7-9e01-423f-accb-29edf6387056 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 202 len: 0 microversion: 2.1 time: 120.088615 Sep 19 16:23:07.577080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 601/1220] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:21:07 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 0 bytes in 120091 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:23:07.588579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:07.589938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:07.602743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.603843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.604758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.606220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.607180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.607930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.707386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:07.719259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ca52c85e-3f29-4255-ae88-31f06570c948 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.140554 Sep 19 16:23:07.720050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 602/1221] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:07 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 142 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:07.728302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:07.730755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:07.737988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.738521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.738964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.740226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.740834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.741491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.776843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6a007719-50d1-4eac-b85e-de681f5797b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.055275 Sep 19 16:23:07.777497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 603/1222] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:07.900613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:07.904507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:07.913944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.914926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.915666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.916537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.917184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.917675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.960377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-89c30458-952c-47f6-8c81-34a22c9617b0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.181506 Sep 19 16:23:07.967051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 604/1223] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:07 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:07.967367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:07.968063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:07.977211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.977599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.978320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:07.979218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:07.979733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:07.980240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:08.008911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:23:08.115474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ee331a15-045c-4fdd-ba0c-e4d3bd87945e tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 204 len: 0 microversion: 2.1 time: 0.151817 Sep 19 16:23:08.116609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 605/1224] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:23:07 2022] DELETE /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:23:08.125303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:08.126153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:08.135773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:08.136426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:08.136972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:08.137831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:08.138793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:08.139202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:08.190440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:23:15.072997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:23:15.105268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:15.108943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:23:15.114319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:23:15.115036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:23:15.116165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.117381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:15.118447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) Sep 19 16:23:15.119647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.120754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-92ddbd98-3e52-4af8-be4f-edf621199205) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:15.120754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-706ba3e6-87a0-4ade-b92d-febeff0cba5a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 67.565965 Sep 19 16:23:15.120754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 620/1225] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:22:07 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 67567 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:23:15.120754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:15.121071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:15.126181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.126869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.127146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.127844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.128288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.128701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.144023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 HTTP/1.1 Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:15.146602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:15.150149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'cf7e835d-ed2f-4b40-a16b-7e12ebd67527'} Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:15.152034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:15.155445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:15.157270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:15.157270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-88d6a9b7-592d-4b92-9669-a1784bfc36e0 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 404 len: 111 microversion: 2.1 time: 0.035766 Sep 19 16:23:15.157270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 621/1226] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:15 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:15.157270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:15.157270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:15.160626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.161301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.161744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.162492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.162956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.163379 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.246283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:15.255152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e0ebe45-06a9-42fe-9fe3-9fbb96eb5268 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.105600 Sep 19 16:23:15.256516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 622/1227] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:15 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:15.265432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:15.269524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:15.281197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.282093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.282927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.284194 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.285028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.285795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.332529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-203d3342-8624-4820-aa35-dcc568bbd7e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.074590 Sep 19 16:23:15.333618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 623/1228] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:15.338643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:15.341027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:15.347315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.347826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.348242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.348998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.349955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.350405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.382707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-37de2b4c-7a46-49d5-9a58-100c58d4d044 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047935 Sep 19 16:23:15.383310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 624/1229] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:15 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:15.388129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cf7e835d-ed2f-4b40-a16b-7e12ebd67527", "name": "network-vif-deleted", "tag": "b963629b-7dec-4fd4-9bfe-f02f8aece1c2"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:23:15.398256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:15.398783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:15.399203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:15.413531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] Creating event network-vif-deleted:b963629b-7dec-4fd4-9bfe-f02f8aece1c2 for instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 on hv-cinder-85837 Sep 19 16:23:15.418934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-24e8e486-f888-4727-95b8-6b4873b5677d req-7b0f3115-16f1-47dc-928a-472e52702666 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.034326 Sep 19 16:23:15.419547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 625/1230] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:23:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:15.425150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-db63241c-2c8c-4170-b83e-3533094d2ae7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '.version_select at 0x7fadbf941b80>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:15.449646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-db63241c-2c8c-4170-b83e-3533094d2ae7 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-27961422" status: 202 len: 0 microversion: 2.1 time: 0.028820 Sep 19 16:23:15.450220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 626/1231] 10.222.0.57 () {58 vars in 1215 bytes} [Mon Sep 19 16:23:15 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-27961422 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:23:16.155308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.156009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.162913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.163395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.163825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.164532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.164977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.165395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.180263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 HTTP/1.1 Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.182089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.183589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'cf7e835d-ed2f-4b40-a16b-7e12ebd67527'} Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.184338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.185639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.186925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.186925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0995687d-6679-42a1-9fe7-3c1979beb401 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 404 len: 111 microversion: 2.1 time: 0.032335 Sep 19 16:23:16.186925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 627/1232] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:23:16 2022] DELETE /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:16.192598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.193212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: cf7e835d-ed2f-4b40-a16b-7e12ebd67527] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.199315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.199799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.200239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.200936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.201383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.201806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.216574 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 HTTP/1.1 Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.218151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.219749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'cf7e835d-ed2f-4b40-a16b-7e12ebd67527'} Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.220677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. Sep 19 16:23:16.222177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.223570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance cf7e835d-ed2f-4b40-a16b-7e12ebd67527 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.223570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-46e93418-81a1-4adb-bbbe-834e32d82d83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527" status: 404 len: 111 microversion: 2.1 time: 0.030998 Sep 19 16:23:16.223570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 628/1233] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/cf7e835d-ed2f-4b40-a16b-7e12ebd67527 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:16.229147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.229653 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.235949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.236439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.236878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.237590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.238042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.238501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.253876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 HTTP/1.1 Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.255571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.257952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'a34c2a71-df97-4e4d-ab68-da770b22a036'} Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.259328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.261781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.263983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.263983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-498db14b-8ca6-4a4d-9ffc-82a29fc2f29c tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 404 len: 111 microversion: 2.1 time: 0.032441 Sep 19 16:23:16.263983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 629/1234] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:23:16 2022] DELETE /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:16.265911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.266565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: a34c2a71-df97-4e4d-ab68-da770b22a036] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.273392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.273873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.274306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.275054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.275502 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.275930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.291340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 HTTP/1.1 Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.292842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.295049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'a34c2a71-df97-4e4d-ab68-da770b22a036'} Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.295950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. Sep 19 16:23:16.297933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.299433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance a34c2a71-df97-4e4d-ab68-da770b22a036 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.299433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e47e97bd-ccac-4437-84c7-532032d002d5 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036" status: 404 len: 111 microversion: 2.1 time: 0.032339 Sep 19 16:23:16.299433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 630/1235] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/a34c2a71-df97-4e4d-ab68-da770b22a036 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:23:16.300146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.300763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.307130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.307630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.308514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.308768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.309284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.309633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.376584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:16.385025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-54b10fd4-3d06-4e02-90af-b398908e4eaa tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.089201 Sep 19 16:23:16.385595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 631/1236] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:16.390425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.391127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.397837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.398339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.398804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.399536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.400145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.400577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.414864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e HTTP/1.1 Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.416437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.417956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'fa7d958b-1b74-4d02-9a97-aae5da88ce3e'} Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.418890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.420143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.421400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.421400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3fa95019-0a8c-4e63-9e1f-7370f2019f83 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 404 len: 111 microversion: 2.1 time: 0.031311 Sep 19 16:23:16.421400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 632/1237] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:23:16 2022] DELETE /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:16.424291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.426555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.432838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.433324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.433757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.434487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.434940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.435365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.467785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-187f1e5a-bf97-4311-b444-cbc7751b83d6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048330 Sep 19 16:23:16.468361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 633/1238] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:16.473246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.475559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.481815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.482305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.482791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.483484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.483940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.484362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.518300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6f9fe2fe-8e9b-48ca-af68-8a9f34eddfba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048865 Sep 19 16:23:16.518917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 634/1239] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:16.524323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:16.524943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: fa7d958b-1b74-4d02-9a97-aae5da88ce3e] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:16.531201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.531690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.532126 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.532831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:16.533288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:16.533710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:16.549218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e HTTP/1.1 Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:23:16.550730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.553161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'fa7d958b-1b74-4d02-9a97-aae5da88ce3e'} Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:23:16.554643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. Sep 19 16:23:16.557377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:23:16.559602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance fa7d958b-1b74-4d02-9a97-aae5da88ce3e could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:23:16.559602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-26677594-56ce-41bc-bbfa-398beeb0d9c1 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e" status: 404 len: 111 microversion: 2.1 time: 0.032299 Sep 19 16:23:16.559602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 635/1240] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:16 2022] GET /compute/v2.1/servers/fa7d958b-1b74-4d02-9a97-aae5da88ce3e => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:23:17.410048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:17.410796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:17.418864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.419344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.419776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.420471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.420915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.421333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.491589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:17.500030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-47deb0fb-7505-4b98-83d3-e356dba6e124 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094134 Sep 19 16:23:17.500747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 636/1241] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:17 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:17.506003 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:17.508403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:17.516537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.517265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.517706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.518434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.518885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.519305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.550111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6008964a-8302-442d-8c9a-dad37e47c132 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048303 Sep 19 16:23:17.550699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 637/1242] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:17 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:17.555950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:17.558082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:17.564645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.565129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.565557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.566250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:17.566736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:17.567149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:17.598833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ae179580-b92a-485d-8664-e663d4304b5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046954 Sep 19 16:23:17.600552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 638/1243] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:17 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:18.131450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=15,nova_api:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:18.533575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:18.534527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:18.541676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.542156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.542624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.543328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.543843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.544271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.611537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:18.619931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-57a8b119-21c3-462b-83da-b402905a0497 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090515 Sep 19 16:23:18.620515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 639/1244] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:18 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:18.625330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:18.627573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:18.634609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.635091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.635521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.636214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.636667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.637084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.669410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f096e728-19d0-4deb-ae4d-ab8b03352425 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047800 Sep 19 16:23:18.669989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 640/1245] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:18.675044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:18.677530 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:18.684264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.684655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.685085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.685786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:18.686234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:18.686693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:18.718304 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3ecb3aab-efe1-43a2-a58b-fb21bef490a1 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047216 Sep 19 16:23:18.718924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 641/1246] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:18 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:19.648232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:19.649044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:19.659558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.660418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.661193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.662479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.663312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.664076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.743532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:19.753058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d40928c2-f906-4de9-91bf-48fe5e2e3a0e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.112434 Sep 19 16:23:19.753655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 642/1247] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:19 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:19.759391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:19.761642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:19.773010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.774617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.775422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.776793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.777635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.778161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.783182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:19.813676 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bfcff73b-f53c-4bf3-81ac-106b0499beb8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058889 Sep 19 16:23:19.814331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 643/1248] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:19.820030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:19.822622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:19.828989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.829471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.829904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.830649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:19.831095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:19.831518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:19.867356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a4cd5843-569f-4348-9773-ebea79dce6fd tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051915 Sep 19 16:23:19.867938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 644/1249] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:19 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:20.785950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:20.786740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:20.794703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.795184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.795620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.796332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.796781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.797205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.862808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:20.871829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dfb40ac5-6485-4b53-90a8-001b05f1c6a1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090316 Sep 19 16:23:20.872434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 645/1250] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:20 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:20.877442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:20.879704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:20.886071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.886601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.887045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.887760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.888216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.888682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.920482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-97b4d4f5-8516-450e-a8cb-3a6db5f0e31c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046877 Sep 19 16:23:20.921059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 646/1251] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:20.925958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:20.928793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:20.934889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.935303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.935743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.936445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:20.936905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:20.937335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:20.967203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-67d66db2-51ee-48f2-b7e9-fc73427a4c8b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045096 Sep 19 16:23:20.967783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 647/1252] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:20 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:21.896312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:21.896991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:21.903479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:21.903960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:21.904398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:21.905100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:21.905544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:21.905973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:21.972982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:21.981315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4c160856-2a59-49d3-91f1-2c49f45a4f74 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088488 Sep 19 16:23:21.981543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 648/1253] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:21 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:21.987048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:21.989845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:21.995805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:21.996282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:21.996715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:21.997410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:21.997854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:21.998273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:22.030033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5dbf3f30-11c1-4492-b1de-de68c3ba76c3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.047452 Sep 19 16:23:22.030765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 649/1254] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:22.035472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:22.037651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:22.044892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:22.044892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:22.044892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:22.045474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:22.045867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:22.046287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:22.089820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed91e818-edcf-405b-a7e1-9b86614d7586 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057888 Sep 19 16:23:22.090634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 650/1255] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:22 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:23.003477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:23.004206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:23.013395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.014219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.015012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.016352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.017176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.017948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.102609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:23.111837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6c842516-5e47-4bb4-bfff-b49a29611885 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.112254 Sep 19 16:23:23.112471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 651/1256] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:22 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:23.117837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:23.120087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:23.126510 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.126996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.127429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.128133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.128588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.129012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.160109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f86fefd4-706c-4aad-bdd3-8b90559ad4f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046545 Sep 19 16:23:23.160701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 652/1257] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:23.165902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:23.168103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:23.174483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.174969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.175403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.176107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:23.176560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:23.177526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:23.209476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a08eb240-0082-4db2-87ae-ece472783507 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047214 Sep 19 16:23:23.210056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 653/1258] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:23 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:24.152752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:24.153553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:24.160280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.160773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.161275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.162068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.162713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.163149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.232050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:24.241164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-93e8ba6c-c35a-40e3-9526-fe1b11a700b1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092780 Sep 19 16:23:24.241753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 654/1259] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:24 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:24.247026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:24.249269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:24.256552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.257044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.257481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.258525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.258990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.259414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.292457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6ac60c50-b638-481c-b9a7-f9fe0a1760ab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049468 Sep 19 16:23:24.293028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 655/1260] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:24 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:24.298079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:24.300254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:24.306946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.307430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.307867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.308564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:24.309017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:24.309440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:24.340599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6fe19e53-330f-42f4-ab67-03afb60e8613 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046504 Sep 19 16:23:24.341185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 656/1261] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:24 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:25.078277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=41,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:25.267369 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:25.268140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:25.275218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.275924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.276406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.277177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.277773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.278294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.346046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:25.355386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a63126c2-53c7-4ff4-8dac-7850b225881e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091895 Sep 19 16:23:25.355980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 657/1262] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:25 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:25.361205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:25.363459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:25.370614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.371106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.371549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.372262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.372716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.373142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.405613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-61356421-ade9-4d6f-a318-0278a848c95c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048445 Sep 19 16:23:25.406210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 658/1263] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:25.411983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:25.414205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:25.420646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.421120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.421560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.422282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:25.423021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:25.423449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:25.452132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:25.458273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8ab11354-db20-434e-9222-f23b5a2eec7b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050841 Sep 19 16:23:25.459058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 659/1264] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:25 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:26.381209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:26.381875 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:26.388716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.389203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.389646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.390356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.390840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.391269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.463890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:26.477470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4289498e-cacb-48ef-a37d-42945dfa441c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100394 Sep 19 16:23:26.478440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 660/1265] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:26 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:26.486783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:26.491250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:26.501463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.502302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.503043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.504147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.505542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.506220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.541695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1001105f-bc12-4f0f-a7d0-e17e93c65654 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.061794 Sep 19 16:23:26.542475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 661/1266] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:26 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:26.548457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:26.550923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:26.558358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.558952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.559439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.560209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:26.560717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:26.561190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:26.595329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a976b8e-6fdd-4f1a-b66d-9ae8b7697d49 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051567 Sep 19 16:23:26.595938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 662/1267] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:26 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:27.504954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:27.505722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:27.512611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.513107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.513537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.514256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.514818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.515248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.584131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:27.592753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e48c2f50-29b5-4f64-a62d-a1870faeb52a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092127 Sep 19 16:23:27.593519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 663/1268] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:27 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:27.598706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:27.600868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:27.609011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.609495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.609932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.610667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.611120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.611549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.644245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e6696caa-e055-4a8a-951f-d611a96aeddf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049516 Sep 19 16:23:27.644833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 664/1269] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:27.650321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:27.652524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:27.658799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.659285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.659725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.660428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:27.660918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:27.661346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:27.695162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0de7af78-7e41-4123-8a55-cda2fb905b02 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048681 Sep 19 16:23:27.695760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 665/1270] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:27 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:28.631027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:28.631727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:28.638743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.639229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.639669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.640389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.640844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.641290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.713415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:28.722471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1c918beb-ee41-4137-bba4-45fd6b81c1d2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095822 Sep 19 16:23:28.723068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 666/1271] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:28 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:28.728941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:28.731571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:28.738245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.738831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.739244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.740014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.740442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.740866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.775410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-09644227-ce54-47c4-8c2d-15c0c44cc339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050552 Sep 19 16:23:28.776155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 667/1272] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:28.783336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:28.785834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:28.793285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.793777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.794214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.795031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:28.795482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:28.795908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:28.829068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-39a39c7b-2904-48a0-b49c-ca52afa091ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051656 Sep 19 16:23:28.829661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 668/1273] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:28 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:29.754357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:29.759789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:29.768542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=99,nova_api:DELETE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:29.776564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.777342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.778103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.779478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.780321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.781071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.876529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:29.885230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d522275d-9843-41ee-bc37-8b7d25026101 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.138419 Sep 19 16:23:29.885844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 669/1274] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:29 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:29.890862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:29.893163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:29.900561 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.901084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.901515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.902240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.902741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.903168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.938897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1010dcd9-54b2-4459-8d09-b4c858c3949b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051829 Sep 19 16:23:29.939697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 670/1275] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:29.945689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:29.948104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:29.954712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.955203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.955671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.956386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:29.956870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:29.957311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:29.995349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-21a48a95-8060-4f14-9b5f-a4d6e19b3a31 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054460 Sep 19 16:23:29.995947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 671/1276] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:29 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:30.913147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:30.913856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:30.920582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:30.921071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:30.921509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:30.922242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:30.922987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:30.923417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:30.991010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:30.999072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-65e5edab-a498-4eef-8db4-7ba3517acfbe tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090544 Sep 19 16:23:30.999655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 672/1277] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:30 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:31.004417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:31.007149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:31.013422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:31.013936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:31.014364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:31.015117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:31.015555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:31.015982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:31.047509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c7febb5e-9da0-4df6-9f11-c5dd72f63dfa tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046781 Sep 19 16:23:31.048334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 673/1278] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:31.053149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:31.055380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:31.061740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:31.062239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:31.062709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:31.063423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:31.063884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:31.064336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:31.102805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-33a13b99-cf0f-488f-bd91-93b7b3abfbb6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052985 Sep 19 16:23:31.103886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 674/1279] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:31 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:32.023534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:32.024352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:32.031153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.031636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.032069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.032786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.033234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.033654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.100389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:32.108901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-54fdacd6-40ef-4275-bf62-e0012b205f64 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.089858 Sep 19 16:23:32.109501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 675/1280] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:32 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:32.115230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:32.117422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:32.125831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.126490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.126986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.127761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.128253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.128734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.147866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:32.163125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-448b16f4-b497-41ce-a87b-f27d3f95f840 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052511 Sep 19 16:23:32.163727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 676/1281] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:32.169075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:32.171440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:32.181335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.182226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.183066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.184376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:32.185197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:32.185994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:32.220930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b533d98-51a4-44f9-acf6-0b75719ac3c0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056116 Sep 19 16:23:32.221557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 677/1282] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:32 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:33.135224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:33.135911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:33.142773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.143262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.143701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.144411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.144862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.145791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.211853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:33.219734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-109e8c85-d651-4aa2-ade0-e60a78a112ff tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088400 Sep 19 16:23:33.220299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 678/1283] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:33 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:33.225471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:33.227645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:33.233953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.234471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.234913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.235626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.236082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.236513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.268157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7ceb6858-927b-408c-8f1b-5cfa83990c50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046785 Sep 19 16:23:33.268770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 679/1284] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:33.274038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:33.276259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:33.282356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.282870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.283309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.284040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:33.284514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:33.284959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:33.317170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d3a2703a-619f-4778-aae4-cc94c4062c19 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047325 Sep 19 16:23:33.317746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 680/1285] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:33 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:34.244200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:34.245270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:34.251367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.251848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.252280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.252980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.253429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.253855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.322078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:34.330980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0a47cdd4-79d4-4b3f-9a58-a0684bd0c033 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090606 Sep 19 16:23:34.331707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 681/1286] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:34 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:34.336900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:34.339377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:34.346255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.346907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.347503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.348319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.348955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.349474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.383090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5895e367-d7d8-47b7-a42f-5a02ec04eed0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050205 Sep 19 16:23:34.383684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 682/1287] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:34 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:34.388447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:34.390631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:34.396873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.397355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.397789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.398546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:34.398996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:34.399419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:34.433202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b772ecee-6ab6-407c-a9e9-9688d060608c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048420 Sep 19 16:23:34.433777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 683/1288] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:34 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:35.355509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:35.356275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:35.363173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.363686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.364171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.364906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.365377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.365805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.435364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:35.444136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ef0c5e1-b100-4cf5-8d61-0ccfdaa3252a tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092539 Sep 19 16:23:35.444726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 684/1289] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:35 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:35.450897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:35.453140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:35.460380 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.460862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.461290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.461999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.462487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.462915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.501396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d0215d90-cd08-4245-84b0-a121a93f576b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054788 Sep 19 16:23:35.501975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 685/1290] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:35.507180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:35.509309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:35.515275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.515755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.516200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.516875 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:35.517316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:35.517768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:35.549579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d0b733cb-45de-4862-a385-4937c19ad3d6 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046495 Sep 19 16:23:35.550185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 686/1291] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:35 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:36.468601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:36.469352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:36.476636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.477117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.477547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.478246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.478728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.479152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.546548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:36.556390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3caf4e96-71f8-42b9-9875-aaeef9c15249 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.091728 Sep 19 16:23:36.570442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 687/1292] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:36 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:36.570442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:36.570442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:36.582055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.582652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.583132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.583881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.584376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.584969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.623735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-24597306-1ce8-4911-9481-3490d963f506 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.064992 Sep 19 16:23:36.624453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 688/1293] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:36.630784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:36.633102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:36.639744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.640368 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.640858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.641765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:36.642314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:36.642863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:36.677289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8b776d76-0924-418a-be2d-bf17cf1ed454 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050926 Sep 19 16:23:36.677891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 689/1294] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:36 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:37.583706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:37.584489 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:37.591431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.591983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.592554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.593346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.593849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.594341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.665976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:37.674903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-981ecedd-22e7-4609-a2f9-13b0ec6f8cd1 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095595 Sep 19 16:23:37.675572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 690/1295] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:37 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:37.684599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:37.688619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:37.699489 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.700357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.701145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.702467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.703283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.704060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.742027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f3a57ca7-7a34-440f-b3f5-fcc07d501ef1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065007 Sep 19 16:23:37.742654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 691/1296] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:37.747769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:37.750216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:37.757132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.757625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.758067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.758815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:37.759269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:37.759689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:37.795151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-406037f3-81a3-4776-b1af-b5a9d3407e21 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051314 Sep 19 16:23:37.795980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 692/1297] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:37 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:38.713376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:38.714085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:38.721623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.722158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.722684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.723975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.724465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.724929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.729801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:38.800976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:38.809899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-edad2f4a-78b8-4d72-a531-11d64bdb3054 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100404 Sep 19 16:23:38.810512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 693/1298] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:38 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:38.815474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:38.817689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:38.824101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.824611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.825029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.825736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.826233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.826679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.860191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad2daf6d-c474-4517-a457-8ccf017604e5 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048582 Sep 19 16:23:38.860783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 694/1299] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:38.866256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:38.868537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:38.875159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.875712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.876356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.877181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:38.877738 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:38.878268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:38.919249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3ea5d013-e687-4944-a9a3-74cab5dc0e3a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057240 Sep 19 16:23:38.920131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 695/1300] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:38 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:39.850209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:39.851007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:39.858082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:39.858712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:39.859278 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:39.860108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:39.860607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:39.861071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:39.959189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:39.967871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-842f06f1-3af7-478d-9fec-6ec51e82aed2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.122742 Sep 19 16:23:39.968490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 696/1301] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:39 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:39.976474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:39.978813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:39.985716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:39.986124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:39.986555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:39.987305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:39.987755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:39.989246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:40.022742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-48f7c708-2b1a-4f44-9f28-c6b4290f5f81 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052276 Sep 19 16:23:40.023497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 697/1302] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:40.031883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:40.034125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:40.041427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:40.041927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:40.042428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:40.043175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:40.043632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:40.044053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:40.077739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ca6d78d3-72b6-435f-9b70-35d95457f65f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052810 Sep 19 16:23:40.078323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 698/1303] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:40 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:40.851312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:23:40.851970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:23:40 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:23:40.998235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:40.998913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:41.005866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.006421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.006918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.007770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.008367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.008826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.076938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:41.086406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f3a3d5e4-0f01-4962-a59c-8a75e110abe8 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092207 Sep 19 16:23:41.087065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 699/1304] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:40 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:41.092325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:41.094903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:41.103071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.103631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.104131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.104902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.105435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.105948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.143150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b8318210-5f4e-4b43-9b6a-b7a934f2bf7b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.054958 Sep 19 16:23:41.143733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 700/1305] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:41.148512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:41.150725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:41.156850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.157336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.157767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.158490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:41.158943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:41.159356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:41.192620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-33f88880-00b0-479c-b040-3d281a795c37 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047804 Sep 19 16:23:41.193232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 701/1306] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:41 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:42.115334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:42.116205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:42.123281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.123771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.124211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.124968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.125423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.125922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.195462 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:42.204563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-703132d0-bd1c-417b-92dd-b98f3d75757d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094507 Sep 19 16:23:42.205218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 702/1307] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:42 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:42.210533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:42.212883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:42.219906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.220403 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.220848 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.221571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.222032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.222495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.257254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c7a9de17-a227-4bf7-8e72-3a6b49536f88 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050830 Sep 19 16:23:42.257849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 703/1308] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:42 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:42.262847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:42.265131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:42.271529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.272019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.272461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.273186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:42.273643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:42.274071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:42.307896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-50cccd0a-8a5e-474d-89a6-e09cfa7f2b35 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048860 Sep 19 16:23:42.308405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 704/1309] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:42 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:43.244340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:43.245474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:43.257723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.258655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.259452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.260775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.261617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.262445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.346202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:43.355156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6996e6bb-85da-4e1f-b5e3-6dbdbf5046ae tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.119038 Sep 19 16:23:43.355760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 705/1310] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:43 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:43.360873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:43.363140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:43.370139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.370675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.371112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.371823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.372277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.372707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.405094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f84bf850-f17b-4d49-91cf-41a70b96416c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048220 Sep 19 16:23:43.405693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 706/1311] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:43.411264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:43.413636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:43.420018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.420511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.420953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.421671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:43.422126 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:43.422585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:43.460892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a9d4822-1a4a-400a-98ca-6b6f3e8e743e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053365 Sep 19 16:23:43.461670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 707/1312] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:43 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:44.395437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:44.396136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:44.402950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.403435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.403880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.404609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.405152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.405586 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.481932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:44.491082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a8a21093-afc2-4ad3-a15f-6ab3a86aa31e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.099657 Sep 19 16:23:44.491757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 708/1313] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:44 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:44.501052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:44.505121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:44.516551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.517437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.518211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.519627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.520953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.521503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.552211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:44.558657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2cb80c6-a721-4484-9442-b5506dce8087 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.065420 Sep 19 16:23:44.559378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 709/1314] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:44 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:44.564959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:44.567712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:44.574334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.574852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.575285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.575991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:44.576434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:44.576852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:44.610620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e6d4746c-eea3-4ed6-8859-89bf9b1003e4 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049852 Sep 19 16:23:44.611224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 710/1315] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:44 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:45.529112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:45.529940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:45.537202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.537735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.538219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.539011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.539569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.540091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.604804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:45.613657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f9e5625f-9235-450e-80f6-86077162e18f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.088416 Sep 19 16:23:45.614670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 711/1316] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:45 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:45.619546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:45.621726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:45.628513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.629017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.630020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.630691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.631149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.631571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.668119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5d82360a-1ee3-421a-b7d8-7d3c345d0787 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052344 Sep 19 16:23:45.673482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 712/1317] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:45.678292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:45.680533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:45.686847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.687327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.687759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.688459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:45.688909 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:45.689327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:45.721562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f0a0120b-5176-4fad-9069-d01f71a6b25a tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047956 Sep 19 16:23:45.722155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 713/1318] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:45 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:46.645051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:46.645803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:46.652722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.653182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.653615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.654330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.654802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.655227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.724831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:46.733799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8fcd58f1-1c99-404b-adea-88004d6a5698 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092697 Sep 19 16:23:46.734520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 714/1319] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:46 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:46.739814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:46.742079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:46.748637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.749131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.749569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.750298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.750791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.751217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.793247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c15752f1-ff09-4566-86ad-189231b284fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057319 Sep 19 16:23:46.794218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 715/1320] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:46 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:46.803246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:46.806355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:46.814857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.815641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.816083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.817572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:46.818311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:46.818877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:46.853459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a8bf115c-7af6-4008-806e-c3e68c566a1b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057838 Sep 19 16:23:46.854066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 716/1321] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:46 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:47.761436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:47.762553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:47.771526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.772056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.772534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:47.773568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.774207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.774877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:47.910757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:47.919508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-752f0594-0e0a-4638-af02-a4f522afd1a6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.163677 Sep 19 16:23:47.920153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 717/1322] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:47 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:47.925715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:47.928721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:47.935812 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.936305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.936744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:47.937479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.938043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.938512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:47.971128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-12201d16-b329-41e5-95b2-ff9e709f7b97 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049847 Sep 19 16:23:47.971833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 718/1323] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:47.979144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:47.982414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:47.985571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:47.990038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.990686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.991132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:47.991890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:47.992401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:47.992834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:48.039394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7fbce6ad-0536-40aa-863b-c751f4f2d3cf tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065706 Sep 19 16:23:48.040372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 719/1324] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:47 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:48.948341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:48.949396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:48.956283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:48.956838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:48.957822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:48.958618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:48.959080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:48.959513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:49.033405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:49.042910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4296b8ec-933f-4339-bb39-4e132059e31f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.098961 Sep 19 16:23:49.043576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 720/1325] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:48 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:49.049065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:49.051441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:49.058013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:49.058567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:49.059010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:49.059763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:49.060215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:49.060666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:49.095153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-559a6c51-9f95-4084-9f63-400cff98ac74 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050364 Sep 19 16:23:49.095834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 721/1326] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:49.100985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:49.103464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:49.110142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:49.110672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:49.111296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:49.112071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:49.112562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:49.113041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:49.155903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-821e64fd-66a2-4fe5-9d6c-d7d0c3293b12 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058847 Sep 19 16:23:49.156515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 722/1327] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:49 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:50.069477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:50.070246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:50.076995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.077481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.077920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.078672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.079130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.079555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.166879 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:50.176616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ec40c9c7-5135-4ee1-857c-dc807085ae04 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.111084 Sep 19 16:23:50.177219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 723/1328] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:50 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:50.182712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:50.185386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:50.197073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.197964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.198803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.199854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.200430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.200952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.235518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-36c1582c-e7ad-46be-bf22-51ef83ba13e8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.057176 Sep 19 16:23:50.236107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 724/1329] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:50.241901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:50.244280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:50.251498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.252037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.252552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.253402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:50.253977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:50.255112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:50.291018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-074be40a-94aa-4416-9d56-f7e30c673375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053736 Sep 19 16:23:50.291618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 725/1330] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:50 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:51.208070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:51.208795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:51.216575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.217143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.217675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.218487 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.218939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.219411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.241734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:51.256781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:23:51.257069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:23:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 (ip 10.222.0.57) !!! Sep 19 16:23:51.297993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:51.307104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3277025d-0524-4bfb-8632-01e5a7cdaee2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.103462 Sep 19 16:23:51.307792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 726/1331] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:51 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:51.395468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:51.397713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:51.404784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.405268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.405700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.406465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.406914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.407337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.440035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0cbb3c46-6d0e-41f1-a1e0-1e23fe0fbb23 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.131074 Sep 19 16:23:51.440716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 727/1332] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:51.446944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:51.449467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:51.456178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.456891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.457968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.458895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:51.459490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:51.460080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:51.493131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dc74ace3-02dc-4f09-bdb8-026fa875d655 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050649 Sep 19 16:23:51.493817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 728/1333] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:51 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:52.263476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:23:52.264097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:23:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:23:52.343321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:52.344238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:52.355762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.356205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.356685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.357480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.358082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.358617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.434577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:52.443691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8c2e3731-52d8-4a4b-a8a5-82eb4cc8dea9 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.105846 Sep 19 16:23:52.444281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 729/1334] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:52 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:52.454971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:52.457476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:52.464398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.464995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.465549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.466438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.466944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.467543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.502067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1160b2be-ae17-4da3-90c9-114565e34ba0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051587 Sep 19 16:23:52.502702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 730/1335] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:23:52.508050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:52.510288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:52.516789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.517281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.517720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.518455 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:52.518905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:52.519329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:52.553598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-76cc99d7-b025-48a4-afc2-0e79d3df0879 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049805 Sep 19 16:23:52.554201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 731/1336] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:52 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:53.466927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:53.467627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:53.475234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.475716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.476158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.476852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.477298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.477718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.547062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:53.556111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e4006fe-7426-4909-aa8d-dbcd3c18e9f6 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092966 Sep 19 16:23:53.556669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 732/1337] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:53 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:53.561661 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:53.563841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:53.570677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.571152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.571589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.572288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.572731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.573152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.606016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e7fc41c0-54f8-4282-bd74-27b16fbcf769 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048134 Sep 19 16:23:53.606643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 733/1338] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:53.611473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:53.613708 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:53.620035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.620520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.620957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.621674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:53.622120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:53.622590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:53.656671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8b3fb0ac-6a93-4f5c-981f-0f43d84e42be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048930 Sep 19 16:23:53.657271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 734/1339] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:53 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:54.580551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:54.581342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:54.588664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.589148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.589580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.590293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.590782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.591212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.666365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:54.678840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-73850f10-d52f-4c39-b81b-3f385703851d tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102076 Sep 19 16:23:54.679447 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 735/1340] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:54 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:54.684710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:54.686950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:54.694270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.694781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.695213 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.695920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.696367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.696790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.732897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6697a87b-7d21-45ac-93bb-6cc12196e9c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.052326 Sep 19 16:23:54.733500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 736/1341] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:54 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:54.739503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:54.741813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:54.748383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.748898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.749308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.750050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:54.750677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:54.751113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:54.787898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d98cefb9-a18e-4538-a8f7-7de9eaf78329 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053211 Sep 19 16:23:54.788493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 737/1342] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:54 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:55.717646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:55.718872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:55.726980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.727674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.728211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.729023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.729576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.730087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.805205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:55.813931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3427718a-a790-4b30-82b5-19d9e859ae1e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.100514 Sep 19 16:23:55.814606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 738/1343] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:55 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:55.820312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:55.822656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:55.829545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.830038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.830484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.831297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.831741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.832746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.866289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-26dc9244-ac84-4458-8ccd-4ab781df84cf tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050431 Sep 19 16:23:55.866894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 739/1344] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:55.871712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:55.873897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:55.880959 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.881443 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.881873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.882623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:55.883069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:55.883490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:55.917524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cb02910d-81fc-46aa-b919-ef240d9e4f04 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049556 Sep 19 16:23:55.918135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 740/1345] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:55 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:56.843304 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:56.844391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:56.855735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:56.856419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:56.857079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:56.858002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:56.858711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:56.859147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:56.948042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:56.957331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3700c319-0813-4ba4-9016-ef8a806dfa95 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.120932 Sep 19 16:23:56.957934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 741/1346] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:56 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:23:56.963151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:56.965491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:56.972078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:56.972615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:56.973056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:56.973769 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:56.974229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:56.974684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:57.009605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d107475b-a4c4-4953-ac32-d6d25306474f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050383 Sep 19 16:23:57.010216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 742/1347] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:57.015352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:57.017743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:57.024792 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:57.025281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:57.025723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:57.026470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:57.026925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:57.027351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:57.031825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:23:57.061415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-32eb1b9b-d26f-421b-b2dd-8845a3f209a0 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050091 Sep 19 16:23:57.062011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 743/1348] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:57 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:57.983965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:57.984691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:57.992666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:57.993208 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:57.993768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:57.994584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:57.995088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:57.995565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:58.066698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:58.075292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cb03d770-2a56-4f5b-96d1-cac824674a72 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.095795 Sep 19 16:23:58.075941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 744/1349] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:57 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:58.081405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:58.084414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:58.091199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:58.091761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:58.092255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:58.093148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:58.093695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:58.094228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:58.130490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-735b41f9-6c71-4e71-9839-6a9b1015af2c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053119 Sep 19 16:23:58.131158 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 745/1350] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:58.139519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:58.141846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:58.148303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:58.148785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:58.149225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:58.149990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:58.150474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:58.150902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:58.183348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c02645f1-c69e-46c6-acf3-bc26e19e3478 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050783 Sep 19 16:23:58.184013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 746/1351] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:58 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:23:59.102905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:59.103723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:59.111149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.111723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.112080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.112860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.113478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.114157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.183382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:23:59.192523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b4fc4c4c-0123-4d42-bd4e-5afb1a71fb69 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093663 Sep 19 16:23:59.193164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 747/1352] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:23:59 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:23:59.198428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:59.200660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:59.208063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.208691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.209505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.210851 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.211694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.212482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.252468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3eddc1e2-d796-4967-9943-6416c499a731 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058098 Sep 19 16:23:59.253060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 748/1353] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:23:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:23:59.258662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:23:59.260894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:23:59.267327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.267822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.268257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.268971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:23:59.269949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:23:59.270419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:23:59.303590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-024810e9-abf8-4619-9a8d-f9207e6bee5f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049385 Sep 19 16:23:59.304192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 749/1354] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:23:59 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:00.216772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:00.217492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:00.224680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.225165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.225603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.226343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.226807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.227231 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.294192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:00.303658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d22497c2-84e1-40c8-ac0b-a5207a72accc tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.090721 Sep 19 16:24:00.304240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 750/1355] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:00 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:00.309211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:00.312060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:00.319024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.319517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.319962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.320683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.321140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.321562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.356356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-42a34b8c-b132-4327-9101-37a55c2dfdb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050976 Sep 19 16:24:00.356956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 751/1356] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:00.362042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:00.364283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:00.370523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.371013 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.371459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.372176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:00.372627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:00.373051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:00.407501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-371f4c47-7beb-418f-87d5-7cccf374d2b3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049414 Sep 19 16:24:00.408089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 752/1357] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:00 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:01.355044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:01.356205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:01.362868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.363355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.363788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.364504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.364960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.365385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.433874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:01.443299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c113ee82-c3b6-42ce-b91c-ec4487c21820 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.094395 Sep 19 16:24:01.443968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 753/1358] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:01 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:01.448822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:01.451041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:01.457965 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.458467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.458936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.459654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.460104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.460532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.493810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6bc41a29-4e3c-49c7-93d0-77a38e7d5653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048795 Sep 19 16:24:01.494640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 754/1359] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:01.499467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:01.501724 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:01.508740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.509227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.509663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.510429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:01.510881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:01.511301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:01.544374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b64250a0-a050-47ba-a1be-fe842e8b6b58 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048813 Sep 19 16:24:01.544997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 755/1360] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:01 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:02.482566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:02.483406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:02.490601 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.491190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.491650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.492387 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.492912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.493346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.569782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:02.580671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5df7cadc-ad1b-4d0a-97bd-4ce24a194b1b tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.102386 Sep 19 16:24:02.581269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 756/1361] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:02 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:24:02.586333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:02.588596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:02.595299 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.595797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.596239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.596961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.597470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.597904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.632481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b6b79a3-9c16-4247-bdec-64e7a5db0612 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.049921 Sep 19 16:24:02.633121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 757/1362] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:02.638178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:02.640563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:02.647186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.647677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.648122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.648889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:02.649293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:02.649753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:02.684719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e3771470-daa3-4381-bfe8-3389970d27c3 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050389 Sep 19 16:24:02.685531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 758/1363] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:02 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:03.617776 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:03.618680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:03.625778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.626267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.626732 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.627454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.627911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.628342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.697006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:03.700577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:03.706019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3374200d-8bf0-4a01-85d3-4142a0eeccef tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.092296 Sep 19 16:24:03.706643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 759/1364] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:03 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:03.712351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:03.714618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:03.720930 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.721422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.721862 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.722637 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.723093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.723588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.757266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d192b20b-a70a-4bdb-a57d-d371fa743ae1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.048911 Sep 19 16:24:03.758182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 760/1365] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:03.767112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:03.770180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:03.778658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.779391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.780068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.781085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:03.781662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:03.783178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:03.816642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3461e965-b3c4-4a72-bd09-b383e5282d0f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056460 Sep 19 16:24:03.817233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 761/1366] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:03 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:04.746291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:04.747470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:04.760092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.760952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.761730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.763295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.763878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.764655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.838071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:04.847523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8e216b07-2cd1-4881-b1f1-29355b3e012f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.108096 Sep 19 16:24:04.848264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 762/1367] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:04 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:04.854424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:04.857040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:04.864902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.865402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.865850 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.866614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.867083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.867520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.903063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ccf7b392-12e0-4429-a2ea-a2d647f9fb2d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053445 Sep 19 16:24:04.903831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 763/1368] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:04 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:04.912570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:04.915289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:04.921977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.922513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.923217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.923940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:04.924413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:04.924844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:04.957564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e198beff-471c-4f11-9fbc-83995640442c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052175 Sep 19 16:24:04.958047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 764/1369] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:04 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:05.881031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:05.881966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:05.889023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:05.889513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:05.889964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:05.890748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:05.891211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:05.891643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:05.971122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:05.981151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f25f668a-3ad1-4ed9-aa1b-755253c3ffa2 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.104594 Sep 19 16:24:05.981742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 765/1370] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:05 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:05.986916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:05.989232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:05.996474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:05.996968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:05.997408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:05.998139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:05.998634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:05.999065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:06.033680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3cd12356-52c8-4ea4-811c-29aed0620e12 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.050706 Sep 19 16:24:06.034160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 766/1371] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:06.039644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:06.041854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:06.048752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:06.049242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:06.049674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:06.051007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:06.051463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:06.051888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:06.085396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-783b54b0-805d-409d-a2ef-53b586d68c74 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049574 Sep 19 16:24:06.086000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 767/1372] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:06 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:07.008736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:07.009451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:07.016919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.017495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.018054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.018937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.020042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.020584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.092584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:24:07.101668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-808bd712-9e7f-4169-b52a-14768fa952f5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.097171 Sep 19 16:24:07.102345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 768/1373] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:24:07 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:24:07.108170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:07.110910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:07.116797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=100 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:07.119394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.119793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.120218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.120950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.121411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.121843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.157415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d6f01807-c4b9-4185-bae2-836e51f15040 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.053796 Sep 19 16:24:07.158068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 769/1374] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:24:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:24:07.163497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:07.165840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:07.173243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.173815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.174323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.175122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.176162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.176657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.209967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6c5221db-de9b-4d8c-b633-63d171b9ca23 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050630 Sep 19 16:24:07.210609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 770/1375] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:24:07 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:24:07.780830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:24:07.781361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:24:07.788435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.788923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.789361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:07.790076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:24:07.790568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:24:07.790996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:24:17.785628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:24:17.812040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=70 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:08.457007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:25:08.460409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:25:08.461970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:25:08.463066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:25:08.464151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:08.465306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:25:08.466855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) Sep 19 16:25:08.468304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:25:08.469859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-43980d88-3037-4859-804a-8f6552d2dc17) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:25:08.469859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f3a538cf-d619-4dee-a4f1-2a9197cc912f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 120.343751 Sep 19 16:25:08.469859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 606/1376] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:23:08 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 120345 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:25:08.470679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:08.471445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:08.478622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.479113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.479645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.480455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.480868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.481287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.550072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:25:08.557756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-811c3071-d7a6-4988-9c8b-871767353d5c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.093992 Sep 19 16:25:08.558350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 607/1377] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:25:08 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:25:08.563031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:08.566581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:08.572857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.573377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.573817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.574558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.575018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.575438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.606224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5d2f7c12-72bc-4e4a-88c4-0ab7c3168688 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.046779 Sep 19 16:25:08.606852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 608/1378] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:25:08 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:25:08.611902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:08.614246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:08.620427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.621004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.621341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.622082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.622568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.622998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.655076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bca3a260-73fd-4247-b9d2-aa913ea92cd9 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047225 Sep 19 16:25:08.655792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 609/1379] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:25:08 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:25:08.660612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:08.661004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:08.667094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.667585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.668021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.668732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:08.669836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:08.669836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:08.766309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:25:15.272359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:25:15.298040 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:25:15.301182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:25:15.302813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:25:15.303516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:25:15.304622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:15.305935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:25:15.307204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) Sep 19 16:25:15.308641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:25:15.310640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-142b66ea-81e7-4d25-8738-354cef3de95b) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:25:15.310640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-82062c6d-5ee2-497d-b2dd-0419122e9d3b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 67.636605 Sep 19 16:25:15.310640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 771/1380] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:24:07 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 67639 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:25:15.317827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:15.318872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:15.328493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.329794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.330673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.332276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.333210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.333954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.419607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9fd6c22a61be43658ae87052b867e384', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '2a726afa4ec84ec99a5b1bd08c95a0cd', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:25:15.428326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-20f1ebb2-d23b-4c3c-b408-97a9c8b19eee tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 200 len: 1226 microversion: 2.1 time: 0.122069 Sep 19 16:25:15.428912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 772/1381] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:25:15 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 1226 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:25:15.434088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:15.436450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:15.443538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.444409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.444858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.445573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.446022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.446493 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.489257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f6c2491b-c428-4103-8c6a-607c27b1a4d9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.058919 Sep 19 16:25:15.490105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 773/1382] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:25:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:25:15.503912 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:15.506240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:15.512558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.513044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.513476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.514207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.514684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.515106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.550593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-03fc173d-7278-4c45-a433-d0f01b237324 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052742 Sep 19 16:25:15.551191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 774/1383] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:25:15 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:25:15.556932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:25:15.557439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:25:15.563859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.564141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.564541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:15.565281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:25:15.565755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:25:15.566188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:25:18.665196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:25.291534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:25.561292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=8 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:25.586958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:25:57.223775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:25:57.224418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:25:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:27:00.863278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:27:00.863894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:27:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:27:08.923196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:27:08.925805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:27:08.927351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:27:08.928091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:27:08.929210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:27:08.930928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:27:08.932714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) Sep 19 16:27:08.934271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:27:08.935701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4c3d7694-5076-4207-85fe-dd3e735776b4) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:27:08.935701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-277a7fa7-eb0b-462d-86cb-da16aabe0a76 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 120.270716 Sep 19 16:27:08.935701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 610/1384] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:25:08 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 120272 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:27:08.980656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:27:09.011264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:27:09.011771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:27:09.018664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.019159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.019609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.020314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.020768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.021196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.047133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:27:09.109978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Instance is already in deleting state, ignoring this request Sep 19 16:27:09.110850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5c06d470-73da-431d-ac2b-d11a44ac94ed tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 204 len: 0 microversion: 2.1 time: 0.182097 Sep 19 16:27:09.111417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 611/1385] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:27:08 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:27:09.116091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:27:09.118325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:27:09.123975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.124451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.124882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.125566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.126014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.126462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.211753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dcfd6279-1cd1-41f2-90be-a6e82abe669e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action" status: 200 len: 37892 microversion: 2.20 time: 0.099010 Sep 19 16:27:09.212398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 612/1386] 10.222.0.57 () {62 vars in 1319 bytes} [Mon Sep 19 16:27:09 2022] POST /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5/action => generated 37892 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:27:09.217534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:27:09.220545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:27:09.231676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.232156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.232584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.233279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.233799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.234298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.269670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3174b51f-02a1-4250-acf9-2bfe8049dd22 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056046 Sep 19 16:27:09.270229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 613/1387] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:27:09 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:27:09.274735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:27:09.275219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:27:09.281713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.282112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.282586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:09.283263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:09.283705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:09.284120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:11.270998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:27:11.271837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:27:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 (ip 10.222.0.57) !!! Sep 19 16:27:12.275334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:27:12.275949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:27:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 (ip 10.222.0.57) !!! Sep 19 16:27:15.526619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d0913483-e2e0-4fdd-a195-79d54237a68b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93" status: 202 len: 0 microversion: 2.1 time: 119.973984 Sep 19 16:27:15.527512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 775/1388] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:25:15 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 => generated 0 bytes in 119976 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:27:15.533491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:27:15.534037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:27:15.540831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:15.541267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:15.541713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:15.542512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:27:15.543131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:27:15.543634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:27:15.572773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:27:19.279067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:27:25.540705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:27:25.568256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=9,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:27:25.578511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:28:47.783868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:28:47.784360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:28:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 (ip 10.222.0.57) !!! Sep 19 16:29:09.517283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:09.663093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:09.665284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:09.666763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:09.668153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:09.669921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:09.671184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:09.675747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) Sep 19 16:29:09.677992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:09.682026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-35e403a8-c9f9-4969-9145-e55ab069972b) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:09.682026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f2365f3e-c38c-4d15-8cbc-a9f5acc59021 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 120.395468 Sep 19 16:29:09.682026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 614/1389] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:27:09 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 120397 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:09.682026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:09.682658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:09.682658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.682658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.682658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.683213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.683213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.683487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.710647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:09.760970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Instance is already in deleting state, ignoring this request Sep 19 16:29:09.761818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d435a1ef-769a-498a-adba-2e498b4f4a58 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 204 len: 0 microversion: 2.1 time: 0.093193 Sep 19 16:29:09.762834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 615/1390] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:09 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:09.932883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:09.935295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:09.941470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.941958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.942447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.943159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.943615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.944041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.974450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f2082407-f092-422b-8b7b-fef9b3c1113b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.210377 Sep 19 16:29:09.975242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 616/1391] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:09 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:09.979850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:09.980343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:09.986475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.986940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.987375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.988077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:09.988533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:09.988957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:09.996562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 HTTP/1.1 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:29:10.001798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:10.003435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f', 'id': '3ec19191-d0b9-429b-b5bc-18ac7f02af93'} Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:10.004237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:10.005488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:10.006870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:10.008092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) Sep 19 16:29:10.010264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:10.013159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07d2016c-2067-45e0-bad7-0695ac4c9bfd) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:10.013159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad231c45-48ae-4f85-a229-a6b69508740d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93" status: 400 len: 269 microversion: 2.1 time: 114.475045 Sep 19 16:29:10.013159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 776/1392] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:27:15 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 => generated 269 bytes in 114476 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:10.013159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:10.013913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:10.018864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:10.019350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:10.019790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:10.020488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:10.020940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:10.021497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:15.762831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:15.811781 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:15.934142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:15.944046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:15.945800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:15.946599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:15.947762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.948968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:15.949997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) Sep 19 16:29:15.953442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.956208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-4f04d7a2-36ae-4d4f-9620-13603e7fb160) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:15.956208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ae334c17-80af-459d-972c-5fd0f00d561a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 5.969452 Sep 19 16:29:15.956208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 617/1393] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:09 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 5976 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:15.958945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:15.959309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:15.966304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:15.966917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:15.967387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:15.968143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:15.968629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:15.969057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:15.983885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 HTTP/1.1 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:29:15.986988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:15.990077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f', 'id': '3ec19191-d0b9-429b-b5bc-18ac7f02af93'} Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:15.991553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:15.994399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.996970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:15.999947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) Sep 19 16:29:16.002356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.005100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-d313d9e7-5188-456a-ac37-6a0fcc69e2b8) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.005100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5f5784fb-b88c-4b96-a8e1-8f8493df2358 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93" status: 400 len: 269 microversion: 2.1 time: 5.983832 Sep 19 16:29:16.005100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 777/1394] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:10 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 => generated 269 bytes in 5987 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:16.005615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:16.010409 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.010569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.017495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.017983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.018538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.019155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.019646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.020011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.054906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:16.081997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4d3ee4cb-1055-4a50-b5f6-fa01d883eed3 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 204 len: 0 microversion: 2.1 time: 0.129112 Sep 19 16:29:16.083304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 618/1395] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:15 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:16.088815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.091207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.098913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.099398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.099821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.100524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.100972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.101388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.132823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aab5293a-c507-4b1d-8284-51a78c35dec7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048317 Sep 19 16:29:16.133386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 619/1396] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:16.137054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8c61a1b5-00a3-415f-8a37-f31be06a7ce0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 204 len: 0 microversion: 2.1 time: 0.143236 Sep 19 16:29:16.137631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 778/1397] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:15 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:16.137877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.138352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.142736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.143347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.144207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.144638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.145072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.146139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.146284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.146643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.149053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.149525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.149961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.150750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.151199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.151613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.231250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:16.276471 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:16.305717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:16.308057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:16.309674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.310833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.312670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.313835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.315597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) Sep 19 16:29:16.317901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.320274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-eacd24f6-6e49-4bcd-9cde-661aec41e0d1) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.320274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e4457fbd-6369-4ece-8c63-d44d5d5c1b28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 0.175429 Sep 19 16:29:16.320274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 620/1398] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 177 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:16.320274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.320920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.322832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.323318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.323757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.324493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.324949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.325415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.340805 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 HTTP/1.1 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:29:16.344651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:16.346476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f', 'id': '3ec19191-d0b9-429b-b5bc-18ac7f02af93'} Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.347745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.349161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.350952 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.352419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) Sep 19 16:29:16.353860 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.357454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-07187d20-dfa5-4d94-97c7-6babd0098c99) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.357454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5fff414e-e535-4aea-9e5e-66c27507fec3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93" status: 400 len: 269 microversion: 2.1 time: 0.209251 Sep 19 16:29:16.357454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 779/1399] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 => generated 269 bytes in 210 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:16.359399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.360108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.362574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:16.371516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.372023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.372809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.375217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.375742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.376414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.402537 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:16.419030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Instance is already in deleting state, ignoring this request Sep 19 16:29:16.419912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0f3f50f7-37a7-4827-9504-43e54e31739c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 204 len: 0 microversion: 2.1 time: 0.107602 Sep 19 16:29:16.421319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 621/1400] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:16.425991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.428130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.433799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.434281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.434754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.435444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.435897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.436356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.447396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.compute.api [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Instance is already in deleting state, ignoring this request Sep 19 16:29:16.448222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-83a3fbaf-c541-4071-b5cf-6767b10c489c tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 204 len: 0 microversion: 2.1 time: 0.098207 Sep 19 16:29:16.448782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 780/1401] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:16.453415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.453897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.459963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.460871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.460871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.461569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.462074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.462475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.465353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7e855e47-1bf0-4795-bf98-4617ad5b630b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043123 Sep 19 16:29:16.466263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 622/1402] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:16.471543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.472037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.477788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.478251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.478724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.479426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.479888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.480313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.566249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'services' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:16.612945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:16.647323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 HTTP/1.1 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:29:16.649847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:29:16.651546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': 'fbfe137a-fc97-409e-9473-71a345427ae2', 'id': 'd94f3e7c-f979-4419-884f-8141e3deebc9'} Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.652224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.653457 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.655707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.657385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) Sep 19 16:29:16.658975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.665903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-223e73b0-fd6a-4cb6-85e5-f25853ae7752) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.665903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c3614eb1-84a1-4b6b-9c38-771c13b5b033 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9" status: 400 len: 269 microversion: 2.1 time: 0.215682 Sep 19 16:29:16.666829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 781/1403] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments/d94f3e7c-f979-4419-884f-8141e3deebc9 => generated 269 bytes in 217 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:16.677896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.678620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.686935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.687567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.688108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.688915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.689474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.690000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.709710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 HTTP/1.1 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:16.712156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:29:16.713609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': 'ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f', 'id': '3ec19191-d0b9-429b-b5bc-18ac7f02af93'} Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.714313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:29:16.715453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.716742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.719207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) Sep 19 16:29:16.720477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.722541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-559d4174-5032-4197-a5e1-90253f62b152) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.722541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1bb6d20a-9047-4862-8ee6-52b733fb123f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93" status: 400 len: 269 microversion: 2.1 time: 0.246564 Sep 19 16:29:16.722541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 623/1404] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f/os-volume_attachments/3ec19191-d0b9-429b-b5bc-18ac7f02af93 => generated 269 bytes in 249 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:29:16.722945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:29:16.726679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.727187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.737802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.738320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.739255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.739986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.740501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.741180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.757072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 HTTP/1.1 Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:16.758940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.760362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'f5c5064d-5dce-44c2-8610-006177ddd542'} Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.761192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.762460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.764156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.764156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-65655b37-da14-4d3b-a21e-0e2e69f15d68 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 404 len: 111 microversion: 2.1 time: 0.038790 Sep 19 16:29:16.766254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 624/1405] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 111 bytes in 50 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:29:16.773329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.776380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.785269 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.compute.api [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Instance is already in deleting state, ignoring this request Sep 19 16:29:16.785542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.786456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1987ef95-f4b2-4320-8cfc-09fea7cb7922 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 204 len: 0 microversion: 2.1 time: 0.118374 Sep 19 16:29:16.791190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 782/1406] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:29:16 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:29:16.794810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.795599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.796469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.796939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.797314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] [instance: f5c5064d-5dce-44c2-8610-006177ddd542] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.797934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.798607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.804383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.804872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.805317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.806028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.806518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.806945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.822804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 HTTP/1.1 Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:29:16.824358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.825970 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'f5c5064d-5dce-44c2-8610-006177ddd542'} Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:16.827425 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. Sep 19 16:29:16.828646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:29:16.829830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Instance f5c5064d-5dce-44c2-8610-006177ddd542 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:16.829830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b8b8aaff-7105-46f9-b428-0873aa9cf8da tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542" status: 404 len: 111 microversion: 2.1 time: 0.033541 Sep 19 16:29:16.829830 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 783/1407] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:29:16 2022] GET /compute/v2.1/servers/f5c5064d-5dce-44c2-8610-006177ddd542 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:29:16.832727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3a0a6da-f53a-45db-b00c-1e31decb5c97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.065118 Sep 19 16:29:16.833560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 625/1408] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:16.841725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:16.843042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:16.849451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.849903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.850553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:16.851424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:16.852128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:16.852836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.190355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:29:17.198037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-374caee1-043b-4736-a8c9-b65b0ef0fff6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1673 microversion: 2.1 time: 0.368941 Sep 19 16:29:17.198639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 784/1409] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:29:16 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1673 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:29:17.478090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:29:17.480294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:17.487039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.487517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.487951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.488638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.489087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.489510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.570039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cd46ca13-3939-4cf1-b2f1-bb5ed686d7be tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action" status: 200 len: 83091 microversion: 2.1 time: 0.095493 Sep 19 16:29:17.570660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 626/1410] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:29:17 2022] POST /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/action => generated 83091 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:29:17.585180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:17.586105 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:17.596986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.597583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.598220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.599079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.599890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.600744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.846156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:17.848379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:17.855863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.856396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.856795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.857528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:17.858506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:17.858994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:17.864654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:17.895656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cdd305e1-8df6-47cd-b5a1-8a53ad6ccb6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053933 Sep 19 16:29:17.896432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 627/1411] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:18.192999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4af93002-f79c-48e0-9a49-a348edf588c8", "name": "network-vif-deleted", "tag": "60258110-cac3-4864-a18c-34a601420154"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:29:18.203101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.203835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.204562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.216720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] Creating event network-vif-deleted:60258110-cac3-4864-a18c-34a601420154 for instance 4af93002-f79c-48e0-9a49-a348edf588c8 on hv-cinder-85837 Sep 19 16:29:18.222185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-998f5397-c877-466a-b154-104aaa08d22e req-798e3e7c-3489-4c88-8776-2c8ff199393b service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.095450 Sep 19 16:29:18.223109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 628/1412] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:29:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:18.231283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:18.232017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:18.239135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.239625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.240111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.240833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.241283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.241711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.277613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.278102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.278550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.278999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lazy-loading 'fault' on Instance uuid 4af93002-f79c-48e0-9a49-a348edf588c8 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:29:18.328853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '754c1b5cb47746cf93c52d6dc0b8fa01', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '23c705bc001e4228bdfcdfa916a368e9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:29:18.341014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9aed3f12-f506-499c-b8a4-cfe9654d6cfa tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 200 len: 1326 microversion: 2.1 time: 0.113939 Sep 19 16:29:18.341631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 629/1413] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:29:18 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 1326 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:29:18.908570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:18.910794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:18.917694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.918180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.918665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.919404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:18.919825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:18.920251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:18.954663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-02a5cdd7-83a2-4a03-bae8-bff2d56fe961 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050180 Sep 19 16:29:18.955492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 630/1414] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:18 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:19.366504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:19.367153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:19.373150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:19.374281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:19.374746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:19.375434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:19.375889 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:19.376314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:19.389350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:29:19.391040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:29:19.392706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:29:19.393644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:29:19.395160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:29:19.396475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:29:19.396475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2ef525f0-2fdd-4fa2-a690-a741ff61d50c tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.029917 Sep 19 16:29:19.396475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 631/1415] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:29:19 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:29:19.400937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eb3373af-f01f-4c17-8288-f4cf64c47fc7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '.version_select at 0x7fadbf6ec040>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:19.423475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eb3373af-f01f-4c17-8288-f4cf64c47fc7 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2001211246" status: 202 len: 0 microversion: 2.1 time: 0.026008 Sep 19 16:29:19.424029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 632/1416] 10.222.0.57 () {58 vars in 1221 bytes} [Mon Sep 19 16:29:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2001211246 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 19 16:29:19.969032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:19.971315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:19.979415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:19.979903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:19.980342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:19.981035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:19.981487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:19.981904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:20.012125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-70870cf7-ffd7-4b4f-ab9c-54c6cd1a8508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047865 Sep 19 16:29:20.012125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 633/1417] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:21.024472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:21.026692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:21.033607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:21.034032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:21.034511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:21.035215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:21.035663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:21.036095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:21.067055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-10ef30d0-67c2-4ab0-822f-7f0e39fb8e26 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046488 Sep 19 16:29:21.067632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 634/1418] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:21 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:22.079790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:22.081865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:22.088382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:22.088865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:22.089302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:22.090035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:22.090584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:22.091015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:22.120377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bfbe5da5-44a6-404a-904a-46aef0056618 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044683 Sep 19 16:29:22.120930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 635/1419] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:22 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:23.133575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:23.135631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:23.143081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:23.143556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:23.144085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:23.144818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:23.145385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:23.145899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:23.175867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-67e8055d-fdf4-4ce5-8a7e-f717c7fea4fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046317 Sep 19 16:29:23.176455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 636/1420] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:23 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:24.188860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:24.190928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:24.197224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:24.197694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:24.198125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:24.198833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:24.199275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:24.199697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:24.228811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c76a7bfd-e110-4a9f-833f-c73ce40d8eb5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044521 Sep 19 16:29:24.229352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 637/1421] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:24 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:25.241370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:25.243526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:25.249831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:25.250304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:25.250772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:25.251453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:25.251894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:25.252309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:25.283165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d6e326a0-662c-4c4d-b2d6-2a6738abd9c5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045773 Sep 19 16:29:25.283714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 638/1422] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:25 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:26.295701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:26.297833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:26.304520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:26.304989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:26.305419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:26.306095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:26.306576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:26.306997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:26.336260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cf23ffb9-4e16-414a-9da3-e3b577969f09 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044560 Sep 19 16:29:26.336806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 639/1423] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:26 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:27.349369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:27.351628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:27.358070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:27.358589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:27.359029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:27.359710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:27.360158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:27.360581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:27.389445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6633aa02-ae9b-4c56-87da-812162a6bb3c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043502 Sep 19 16:29:27.389994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 640/1424] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:27 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:27.592925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=23 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:27.625041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=3,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:28.401393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:28.403564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:28.410645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:28.411127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:28.411567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:28.412257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:28.412710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:28.413135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:28.442692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9c30aacf-339b-414f-bdfb-f7b702a869f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045317 Sep 19 16:29:28.443178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 641/1425] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:28 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:29.454435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:29.457324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:29.463692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:29.464169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:29.464600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:29.465290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:29.465740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:29.466163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:29.494861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c67d9333-6920-46fa-b0cc-8b0d327d5fcb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044387 Sep 19 16:29:29.495418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 642/1426] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:29 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:30.508018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:30.510162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:30.516658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:30.517129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:30.517561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:30.518240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:30.518710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:30.519138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:30.547765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b5178924-141a-46c3-8730-a20f5b3e84c2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043759 Sep 19 16:29:30.548306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 643/1427] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:30 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:31.563033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:31.566124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:31.575368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:31.576125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:31.577407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:31.578593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:31.579397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:31.580089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:31.609548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-66f4e039-f10a-4642-a2bd-2eefaafc3456 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052919 Sep 19 16:29:31.610113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 644/1428] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:31 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:32.621696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:32.623885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:32.630607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:32.631091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:32.631530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:32.632213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:32.632666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:32.633093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:32.658815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:32.663936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-16e4da73-eb03-4ec0-a41e-0d45bb72e7a6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046276 Sep 19 16:29:32.664500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 645/1429] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:32 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:33.677294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:33.679507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:33.686044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:33.686565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:33.687005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:33.687703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:33.688157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:33.688582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:33.719056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b391da6b-5c9e-4a5b-b0a0-3148c35b04ab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046425 Sep 19 16:29:33.719629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 646/1430] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:33 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:34.731281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:34.733433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:34.739925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:34.740403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:34.740880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:34.741563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:34.742014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:34.742471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:34.772519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f6c87383-75fa-4a85-8fba-cfe6d3454e69 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045190 Sep 19 16:29:34.773088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 647/1431] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:34 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:35.785667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:35.787915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:35.794812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:35.795300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:35.795741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:35.796455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:35.796906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:35.797343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:35.830934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0d8c9de5-3991-4224-a85b-14ee4df314dd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048831 Sep 19 16:29:35.831830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 648/1432] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:35 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:36.845081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:36.847402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:36.854063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:36.854599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:36.855039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:36.855766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:36.856223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:36.856655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:36.892230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-53bb616d-997b-4b47-9ef5-96bc68891cd8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051479 Sep 19 16:29:36.893133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 649/1433] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:36 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:37.904856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:37.907673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:37.914728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:37.915211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:37.915648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:37.916357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:37.916807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:37.917229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:37.951962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c761c460-3f20-4a2c-877a-e212f5b64cfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050599 Sep 19 16:29:37.952628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 650/1434] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:37 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:38.967868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:38.970017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:38.976510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:38.976994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:38.977435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:38.978134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:38.978617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:38.979051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:39.010509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8e608002-b822-47ac-85b9-2ee9da75806c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047162 Sep 19 16:29:39.011131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 651/1435] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:38 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:40.023701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:40.026026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:40.028619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=83,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:40.034908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:40.035313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:40.035751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:40.036478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:40.036966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:40.037414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:40.072681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-691d1482-74db-45ed-b534-6fece4ffccb1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053089 Sep 19 16:29:40.073273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 652/1436] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:40 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:41.084989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:41.087971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:41.095193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:41.095698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:41.096093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:41.096815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:41.097269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:41.097695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:41.126946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e8843c7-1999-4b46-89ac-c5bc68ef3bac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045901 Sep 19 16:29:41.127511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 653/1437] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:41 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:42.139531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:42.141894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:42.148635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:42.149209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:42.149556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:42.150257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:42.150739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:42.151162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:42.183828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e5b82e6f-ab75-441d-b4d7-968d97d86ca1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048321 Sep 19 16:29:42.184407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 654/1438] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:42 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:43.196642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:43.198762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:43.205594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:43.206096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:43.206664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:43.207393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:43.207846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:43.208273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:43.238741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ab93892-b9b6-481a-80fd-c926f5c5010e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046181 Sep 19 16:29:43.239311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 655/1439] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:43 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:44.250736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:44.252934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:44.259458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:44.259942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:44.260375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:44.261074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:44.261523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:44.261942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:44.292057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3470f215-9038-40cd-a2fd-6596050f8901 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045365 Sep 19 16:29:44.292626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 656/1440] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:44 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:45.305423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:45.307615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:45.314724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:45.315206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:45.315642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:45.316344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:45.316792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:45.317219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:45.347982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-12a33850-ca61-4160-b638-493fa6fd4945 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046488 Sep 19 16:29:45.348548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 657/1441] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:45 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:46.361078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:46.363277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:46.369673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:46.370154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:46.370629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:46.371324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:46.371780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:46.372211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:46.402960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-769d8135-1c4d-4e2d-8f8e-7bd64b6698b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046288 Sep 19 16:29:46.403522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 658/1442] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:46 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:47.417437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:47.421354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:47.428399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:47.428895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:47.429337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:47.430058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:47.430561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:47.430991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:47.464765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-30469fe9-9bb0-4b9e-970e-fe9abf4b6a1c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052114 Sep 19 16:29:47.465368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 659/1443] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:47 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:48.478016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:48.480620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:48.487675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:48.488154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:48.488587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:48.489416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:48.489867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:48.490180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:48.524943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f99e404f-fc91-489c-9464-b5ca8a5a5fbe tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050832 Sep 19 16:29:48.525526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 660/1444] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:48 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:49.537633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:49.540186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:49.547301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:49.547782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:49.548214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:49.548917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:49.549360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:49.549778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:49.583184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f236c15b-29ee-40ee-91cf-59400cae1f59 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049562 Sep 19 16:29:49.583773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 661/1445] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:49 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:50.596952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:50.599116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:50.605479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:50.605947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:50.606394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:50.607079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:50.607525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:50.607940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:50.626574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:29:50.641117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-15b93e6c-a0f9-458c-849e-f56777326e34 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048073 Sep 19 16:29:50.641695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 662/1446] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:50 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:51.653650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:51.656578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:51.664039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:51.664668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:51.665144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:51.666080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:51.666688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:51.667229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:51.703606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ae201ced-c132-4709-8f1c-f82db8e09f14 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054012 Sep 19 16:29:51.704200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 663/1447] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:51 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:52.716693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:52.718925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:52.725741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:52.726231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:52.726698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:52.727480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:52.727930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:52.728368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:52.760343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0bd2da63-dad2-42ac-b262-7d0b8fd1917d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047791 Sep 19 16:29:52.760929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 664/1448] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:52 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:53.773341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:53.775589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:53.782362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:53.782877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:53.783322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:53.784024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:53.784472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:53.784898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:53.816795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8987f4ee-f17a-42da-a115-65adc225af5c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047471 Sep 19 16:29:53.817357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 665/1449] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:53 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:54.829648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:54.831927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:54.838435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:54.838921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:54.839361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:54.840098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:54.840529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:54.841065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:54.872977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d39f29f3-6f36-40f1-844c-074ae05e60fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047263 Sep 19 16:29:54.873594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 666/1450] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:54 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:55.885953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:55.888311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:55.895503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:55.895984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:55.896417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:55.897118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:55.897570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:55.897984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:55.930828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9b443160-a177-4dd7-9626-a2b79f055772 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049029 Sep 19 16:29:55.931430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 667/1451] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:55 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:56.944726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:56.947029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:56.954197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:56.954723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:56.955165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:56.955876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:56.956330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:56.956753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:56.991873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-01fd8346-ed84-48ce-bd1d-a560a2a54ec5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051067 Sep 19 16:29:56.992457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 668/1452] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:56 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:29:58.005519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:58.007838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:58.015413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:58.015905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:58.016351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:58.017069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:58.017528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:58.017959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:58.053090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6ba6a4f2-bac1-42f6-9d1f-5ca07b39e6e3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051763 Sep 19 16:29:58.053596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 669/1453] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:57 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:29:59.069533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:29:59.072249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:29:59.079364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:59.079856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:59.080302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:59.081018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:29:59.081474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:29:59.081903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:29:59.114763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d789ead8-2265-4bf8-9b70-687379fdb8e9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049506 Sep 19 16:29:59.115369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 670/1454] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:29:59 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:00.133112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:00.135566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:00.143802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:00.144294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:00.144734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:00.145454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:00.145940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:00.146397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:00.178165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0decfa4b-aa6d-45db-81ca-af404b6432df tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050817 Sep 19 16:30:00.178794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 671/1455] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:00 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:01.199686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:01.206445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:01.214548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:01.215040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:01.215478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:01.216196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:01.216726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:01.217209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:01.257210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8b75f4d5-c5eb-45ae-80eb-2a010dd543a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067477 Sep 19 16:30:01.257916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 672/1456] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:01 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:02.273263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:02.275650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:02.282692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:02.283185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:02.283627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:02.284340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:02.284793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:02.285213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:02.318079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-994f89a6-d1db-4ffa-a97c-536eec564ccf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049369 Sep 19 16:30:02.318713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 673/1457] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:02 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:03.330938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:03.333743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:03.341017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:03.341495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:03.341929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:03.342671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:03.343123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:03.343543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:03.377228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-57f80841-cd3c-4f3a-8598-e3e140102e97 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050234 Sep 19 16:30:03.377827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 674/1458] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:03 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:04.390478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:04.392679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:04.399325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:04.399803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:04.400234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:04.400931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:04.401381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:04.401805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:04.433318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dae44ecb-2634-40d4-8637-b50070c69fd4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047344 Sep 19 16:30:04.433908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 675/1459] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:04 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:05.446452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:05.448649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:05.455323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:05.455800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:05.456231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:05.456926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:05.457368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:05.457788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:05.495496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7b9af0b3-3388-4ce2-9923-e9c016786f9c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052947 Sep 19 16:30:05.496078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 676/1460] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:05 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:06.508488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:06.510763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:06.517756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:06.518250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:06.518726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:06.519481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:06.519941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:06.520372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:06.552360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-49b120f8-72ed-4b0c-bd82-6f5f94d555d2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047910 Sep 19 16:30:06.552950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 677/1461] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:06 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:07.565147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:07.567383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:07.573997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:07.574519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:07.574963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:07.576376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:07.576833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:07.577262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:07.608362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d5103b95-8977-4ddf-9102-dc084f3e4750 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047145 Sep 19 16:30:07.608949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 678/1462] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:07 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:08.623952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:08.626664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:08.633523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:08.634015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:08.634475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:08.635188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:08.635644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:08.636073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:08.640097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:30:08.667309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3dee5833-1cc3-4a1d-ae7e-3b2504a95aba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049111 Sep 19 16:30:08.667881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 679/1463] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:08 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:30:09.682210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:09.685222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:09.693670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:09.694190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:09.694665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:09.695377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:09.695852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:09.696279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:09.727275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-85d3fb47-640f-4b4f-b226-e5507066291a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050187 Sep 19 16:30:09.730109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 680/1464] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:09 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:10.741218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:10.743405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:10.745839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=58 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:30:10.751603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:10.752088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:10.752548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:10.753253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:10.753709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:10.754144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:10.793283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a00a0a0a-d995-4bd3-8157-2518b4983071 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057407 Sep 19 16:30:10.794406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 681/1465] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:10 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:11.808300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:11.810500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:11.818689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:11.819168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:11.820463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:11.821398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:11.821668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:11.822090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:11.857502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2bce4ccb-0940-4358-9aa3-99b4e109c14c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054684 Sep 19 16:30:11.858095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 682/1466] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:11 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:12.870944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:12.873152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:12.880267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:12.880746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:12.881180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:12.881884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:12.882338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:12.882796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:12.916227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4105f17c-31cd-4993-acaa-6c136b6f73f3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049475 Sep 19 16:30:12.916808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 683/1467] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:13.930365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:13.932588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:13.940127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:13.940606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:13.941036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:13.941738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:13.942184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:13.942637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:13.989714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-59142e7c-00c6-41fc-bfce-229adb07d5c0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.063032 Sep 19 16:30:13.990817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 684/1468] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:13 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:15.006938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:15.009155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:15.016751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:15.017238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:15.017683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:15.018405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:15.018867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:15.019290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:15.052576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-986be460-b733-42b2-90ac-a2ef8a00902f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049702 Sep 19 16:30:15.053083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 685/1469] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:16.069680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:16.071910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:16.079360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:16.080255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:16.081076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:16.082422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:16.082981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:16.083417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:16.129832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0b1c84ee-7974-4315-b3d1-ff1b0ce35d06 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067171 Sep 19 16:30:16.130444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 686/1470] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:30:17.143450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:17.145687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:17.153035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:17.153988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:17.154472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:17.155227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:17.155693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:17.156119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:17.198140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-02e3cfe0-de00-4eff-a878-a2e60d3dc3be tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058598 Sep 19 16:30:17.199157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 687/1471] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:30:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:30:17.645896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:30:17.646415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:30:17.652450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:17.652962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:17.653352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:17.654033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:30:17.654509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:30:17.654935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:30:27.650459 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=15 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:30:27.675024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=56 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:30:31.281927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:30:31.282529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:30:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:30:32.286964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:30:32.287528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:30:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:31:10.129948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:31:10.656002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:31:17.689219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:31:17.691420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:31:17.693165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:31:17.694163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:31:17.696057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.698091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.701254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) Sep 19 16:31:17.702953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.704153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-a86fde66-220c-4728-9f0d-7edf945f020e) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:31:17.704153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0bd9375b-56a4-4000-ab8e-016017fbd2f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 120.112341 Sep 19 16:31:17.704153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 785/1472] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:29:17 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 120113 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:31:17.704153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:17.704754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:17.710967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.711446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.711889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.712581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.713268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.713442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.725796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:31:17.728062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:31:17.729416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:31:17.730255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:31:17.731937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.733201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.734244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) Sep 19 16:31:17.737620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.741057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-16cdeedf-a1ec-41e5-ba0d-d2c457bcd30a) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:31:17.741057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-541f34cc-2f2d-47ec-bec4-3a860a434015 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 60.087961 Sep 19 16:31:17.741057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 688/1473] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:30:17 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 60089 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:31:17.741057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:17.741723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:17.744099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d82514f4-16a8-4923-b14c-5e5ba9285823 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049616 Sep 19 16:31:17.744869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 786/1474] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:31:17.746476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.747544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.747962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.748663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.749317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.749745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.749995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:17.750313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:17.756230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.756775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.757223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.758084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.758641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.758846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.769144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 HTTP/1.1 Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:31:17.783220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.784706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '5d5cfe70-1a06-4c64-91bf-4681779bd260'} Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:31:17.785499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.788230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.789506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:31:17.789506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5f94889b-78db-4881-9e8e-8164d9fc5550 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 404 len: 111 microversion: 2.1 time: 0.040329 Sep 19 16:31:17.789506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 689/1475] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:31:17 2022] DELETE /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:31:17.789506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:17.789506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] [instance: 5d5cfe70-1a06-4c64-91bf-4681779bd260] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:17.790210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.790210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.790210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.790210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:17.791526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:17.792052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:17.808609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] HTTP exception thrown: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Request method failure captured: Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 HTTP/1.1 Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJUw-USnhDwXuWd5WGVGaYY55AndUpV-10vciQkQaqTx9V1zftaEB99TAx4S3iWgSHSLj325luxCHdHrvtPwnxVy90u0O8hPtAm6U4UI2HgN20cuEvWKoj3QHgfpDbeKz9tM2PBJjPt5F_bKTriu4SpNvMqOR30yKCof0_LvOMd02aE Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:31:17.810813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/a1842253d6634f1f8b93bfa5be2ddd5f"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_a1842253d6634f1f8b93bfa5be2ddd5f"}]}] Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: a1842253d6634f1f8b93bfa5be2ddd5f Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-1544698964 Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: fd36089b3c2141218660ab2458376798 Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeNegativeTest-1544698964-project Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.812202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '5d5cfe70-1a06-4c64-91bf-4681779bd260'} Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:31:17.813077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. Sep 19 16:31:17.814963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:31:17.816301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] Returning 404 to user: Instance 5d5cfe70-1a06-4c64-91bf-4681779bd260 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:31:17.816301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ca3bd76c-c4da-4739-81ca-17569db081e6 tempest-AttachVolumeNegativeTest-1544698964 tempest-AttachVolumeNegativeTest-1544698964-project] 10.222.0.57 "GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260" status: 404 len: 111 microversion: 2.1 time: 0.034897 Sep 19 16:31:17.816301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 690/1476] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:31:17 2022] GET /compute/v2.1/servers/5d5cfe70-1a06-4c64-91bf-4681779bd260 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:31:18.757224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:18.759429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:18.767219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:18.767703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:18.768145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:18.768844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:18.769295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:18.769717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:18.801373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69e6cf03-5e0f-4349-82e4-3d44672bc083 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047782 Sep 19 16:31:18.801931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 691/1477] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:18 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:19.813649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:19.816683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:19.823900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:19.824472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:19.824999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:19.825776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:19.826790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:19.827299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:19.861282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a5231484-3fe0-4458-b7b3-c6720f36f8f2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051630 Sep 19 16:31:19.861854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 692/1478] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:20.873374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:20.875577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:20.884707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:20.885565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:20.886866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:20.887791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:20.889081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:20.889553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:20.934085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b4c6d08a-f66e-4e83-a6c3-5df1610c2c43 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.064652 Sep 19 16:31:20.935054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 693/1479] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:21.947106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:21.949089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:21.956249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:21.956756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:21.957198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:21.957899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:21.958355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:21.958807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:21.991140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a57bb8e7-408a-4040-aed6-ec1526947098 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048092 Sep 19 16:31:21.991715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 694/1480] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:21 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:23.004108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:23.006423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:23.012809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:23.013299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:23.013739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:23.014499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:23.014958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:23.015387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:23.044403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9da67ad6-e570-4681-8425-152026a26304 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044561 Sep 19 16:31:23.044999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 695/1481] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:22 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:24.057806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:24.059965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:24.066561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:24.067044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:24.067483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:24.068181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:24.068635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:24.069061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:24.099004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f3ebfa0a-2ab2-4fef-93ec-015c48795a6a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045264 Sep 19 16:31:24.099572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 696/1482] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:24 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:25.112928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:25.115116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:25.121490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:25.123991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:25.123991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:25.124844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:25.124844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:25.125492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:25.156586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e7b91f25-e3af-4f1e-aa67-34f88684708a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048529 Sep 19 16:31:25.157277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 697/1483] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:25 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:31:26.169979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:26.172063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:26.178780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:26.179261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:26.179699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:26.180379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:26.180829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:26.181249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:26.210885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a88e2c75-b1c2-43d7-8486-7692e8b5abad tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044898 Sep 19 16:31:26.211437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 698/1484] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:26 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:27.224060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:27.226215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:27.232662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:27.233143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:27.233578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:27.234262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:27.234745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:27.235168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:27.264296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b3249090-2fd6-4515-ab85-7d5557281a56 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043726 Sep 19 16:31:27.264862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 699/1485] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:27 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:27.755232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:27.780427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=14 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:28.276763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:28.278940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:28.285504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:28.285984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:28.286446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:28.287152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:28.287594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:28.288022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:28.318623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-32fd994d-b781-476e-afeb-24aa1154c733 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045607 Sep 19 16:31:28.319180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 700/1486] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:28 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:31:29.332820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:29.334923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:29.341122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:29.341601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:29.342040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:29.342760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:29.343212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:29.343640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:29.372308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e0ba6e88-6ebd-44e9-9e65-6a1a8080cd0c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043583 Sep 19 16:31:29.372871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 701/1487] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:29 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:30.385285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:30.387412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:30.394298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:30.394811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:30.395257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:30.395945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:30.396395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:30.396822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:30.426868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5be54af6-5e9b-4fc2-9b4a-b1caa758e483 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045578 Sep 19 16:31:30.427415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 702/1488] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:30 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:31.439365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:31.441497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:31.447966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:31.448561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:31.449041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:31.449801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:31.450255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:31.450709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:31.480903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d5e22399-7a71-45f4-835a-c4c3bb3fb3f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045477 Sep 19 16:31:31.481461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 703/1489] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:31 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:32.494022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:32.496208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:32.503413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:32.503893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:32.504335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:32.505026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:32.505479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:32.505903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:32.535088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-73417ca1-0e5e-41af-9a4b-c247f430ad4e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045162 Sep 19 16:31:32.535649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 704/1490] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:32 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:33.547833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:33.550039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:33.556645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:33.557121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:33.557559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:33.558242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:33.558718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:33.559144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:33.588761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-85577197-0003-4e98-a11c-32d3a03ff3ac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044893 Sep 19 16:31:33.589309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 705/1491] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:33 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:34.601353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:34.603532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:34.610399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:34.610885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:34.611315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:34.611990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:34.612436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:34.612857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:34.616358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:34.641667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6fb78296-27af-437d-843e-545f8008d6fd tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044270 Sep 19 16:31:34.642243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 706/1492] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:34 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:35.654209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:35.656395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:35.662836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:35.663318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:35.663755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:35.664442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:35.664894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:35.665319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:35.696531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-996f785a-b092-426e-8fa1-6a0d58d73148 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046200 Sep 19 16:31:35.697097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 707/1493] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:35 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:36.709773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:36.711907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:36.718254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:36.718761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:36.719199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:36.719888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:36.720334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:36.720788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:36.749931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8c07f44d-b806-46ac-bbab-b61866b81b79 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044176 Sep 19 16:31:36.751019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 708/1494] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:36 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:37.763210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:37.765408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:37.772470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:37.772951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:37.773394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:37.774096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:37.774577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:37.775004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:37.808581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-49f408c7-ccb6-479f-92d7-2f40af4669c9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049376 Sep 19 16:31:37.809148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 709/1495] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:37 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:31:38.821018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:38.823260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:38.829630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:38.830112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:38.830584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:38.831277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:38.831726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:38.832149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:38.861947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-04fec421-84b7-4a27-87dc-6bae6a33740a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044962 Sep 19 16:31:38.862538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 710/1496] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:38 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:39.874519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:39.877325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:39.883781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:39.884268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:39.884708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:39.885405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:39.885856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:39.886282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:39.915143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d68810a9-3c28-4ebd-bdeb-eb4d2901834b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044612 Sep 19 16:31:39.915698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 711/1497] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:39 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:40.928101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:40.930175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:40.936526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:40.936999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:40.937434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:40.938115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:40.938597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:40.939033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:40.967959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b834795c-94c3-4e3a-8976-e76c7fd97b99 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044138 Sep 19 16:31:40.968506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 712/1498] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:40 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:41.981070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:41.983258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:41.990304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:41.990817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:41.991258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:41.991950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:41.992406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:41.992836 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:42.021667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6917d078-c586-4b77-b002-99203147e8e4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044575 Sep 19 16:31:42.022235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 713/1499] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:41 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:43.035057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:43.037150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:43.043796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:43.044273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:43.044709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:43.045404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:43.045855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:43.046278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:43.081136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-60073be3-22a9-4a56-bea2-5f207004cb93 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050540 Sep 19 16:31:43.081720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 714/1500] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:43 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:44.093807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:44.096414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:44.103007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:44.103547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:44.103988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:44.104676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:44.105124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:44.105545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:44.135719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5282a3cf-a53e-4873-9b20-6b7160101264 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045983 Sep 19 16:31:44.136291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 715/1501] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:44 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:45.147599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:45.149823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:45.156259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:45.156728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:45.157160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:45.157839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:45.158815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:45.159015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:45.188647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-26bbca61-20a4-4b1a-aecf-1f0bd55472fc tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044967 Sep 19 16:31:45.189763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 716/1502] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:45 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:46.201861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:46.204021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:46.211161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:46.211635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:46.212064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:46.212746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:46.213698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:46.214271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:46.243892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bf86cb49-c3f6-4fe4-85a5-5d2bfacea628 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046101 Sep 19 16:31:46.244458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 717/1503] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:46 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:47.256850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:47.259014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:47.265472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:47.265954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:47.266412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:47.267110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:47.267564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:47.267995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:47.297406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f78d03c4-0e3b-426b-a46a-7e3479eca625 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044595 Sep 19 16:31:47.297961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 718/1504] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:47 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:48.310257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:48.312465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:48.315313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=61 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:48.320536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:48.321013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:48.321450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:48.322134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:48.322617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:48.323040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:48.351188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-305a2e98-f938-4599-9f42-41260ed973a8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044915 Sep 19 16:31:48.351743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 719/1505] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:48 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:49.364268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:49.366410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:49.372843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:49.373316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:49.373749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:49.374460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:49.374909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:49.375331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:49.403878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0696efef-e5cb-49d4-994b-7dc9e4503a40 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044308 Sep 19 16:31:49.404428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 720/1506] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:49 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:50.416976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:50.419121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:50.425439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:50.426166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:50.426627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:50.427302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:50.427743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:50.428192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:50.456985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b87225d4-fb85-44f3-a570-ff9c15d2707d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043928 Sep 19 16:31:50.457525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 721/1507] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:50 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:51.469223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:51.471368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:51.477710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:51.478189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:51.478660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:51.479342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:51.479790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:51.480210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:51.505806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:31:51.511080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5d3f6d1f-bf7a-46a8-8065-b12685b69508 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045845 Sep 19 16:31:51.511642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 722/1508] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:51 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:52.523524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:52.525823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:52.532476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:52.533462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:52.533881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:52.534601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:52.535050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:52.535481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:52.565108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-81910023-03ad-42da-80c1-148b484274c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045537 Sep 19 16:31:52.565666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 723/1509] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:52 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:53.577208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:53.579380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:53.586289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:53.586801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:53.587242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:53.587929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:53.588384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:53.588813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:53.617967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8bc44eae-d26c-4ae0-8577-5a59c53422f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044704 Sep 19 16:31:53.618547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 724/1510] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:53 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:54.630811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:54.632969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:54.639358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:54.639839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:54.640281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:54.640974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:54.641428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:54.641858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:54.672132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3143f7de-1465-4380-956a-85ed0a81a941 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045505 Sep 19 16:31:54.672703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 725/1511] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:54 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:55.685529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:55.687688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:55.693953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:55.694447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:55.694880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:55.695556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:55.695995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:55.696412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:55.727216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d18e8a19-a54e-4ea2-a81e-d28df10a1177 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045577 Sep 19 16:31:55.727773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 726/1512] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:55 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:56.740218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:56.742421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:56.748796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:56.749274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:56.749708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:56.750405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:56.750858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:56.751281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:56.781920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cb7a62d3-1be4-48e0-8973-f68b21a087a9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046134 Sep 19 16:31:56.782553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 727/1513] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:56 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:57.794906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:57.797008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:57.803678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:57.804132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:57.804565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:57.805242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:57.805684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:57.806100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:57.836203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d2cbc0f9-0cd6-489b-822e-75f365129bd9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045347 Sep 19 16:31:57.836761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 728/1514] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:57 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:58.849419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:58.851615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:58.857961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:58.858472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:58.858914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:58.859605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:58.860054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:58.860477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:58.889418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-240ee027-949d-41e6-a6a7-1680428e38ba tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044587 Sep 19 16:31:58.889986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 729/1515] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:58 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:31:59.902867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:31:59.905075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:31:59.911577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:59.912056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:59.912489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:59.913173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:31:59.913622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:31:59.914075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:31:59.943203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b04a3f31-d443-4428-afb1-b856ca71cf73 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044545 Sep 19 16:31:59.943752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 730/1516] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:31:59 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:32:00.958621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:00.961485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:00.964243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:32:00.964388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:32:00 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments (ip 10.222.0.57) !!! Sep 19 16:32:00.971475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:00.971956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:00.972384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:00.973070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:00.973513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:00.973931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:01.004803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dda99772-17ea-49a7-b555-f543c3112ec2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.052772 Sep 19 16:32:01.005353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 731/1517] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:00 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:02.017815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:02.020734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:02.027209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:02.027693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:02.028130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:02.028922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:02.029314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:02.029741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:02.059218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2f147d58-3864-4551-9f34-0d56c54f8271 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045705 Sep 19 16:32:02.059779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 732/1518] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:02 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:03.071702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:03.073858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:03.080394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:03.080881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:03.081324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:03.082029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:03.082523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:03.083443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:03.113377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c76c9ec-4eb1-471a-a248-148718c41554 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045881 Sep 19 16:32:03.113943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 733/1519] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:03 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:04.126131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:04.128235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:04.134748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:04.135225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:04.135665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:04.136349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:04.136802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:04.137317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:04.168278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4b540b4f-d92c-40ef-83ec-5253c2d06db5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046148 Sep 19 16:32:04.168852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 734/1520] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:04 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:05.181221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:05.183420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:05.189840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:05.190316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:05.190786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:05.191503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:05.191955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:05.192382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:05.222134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-23ab6978-23e5-46bd-91a5-2e6e048f26b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046003 Sep 19 16:32:05.222725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 735/1521] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:05 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:06.235706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:06.238019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:06.244473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:06.244956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:06.245386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:06.246072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:06.246575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:06.247004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:06.276012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-600e3bd4-7a3e-4784-a248-eb5d044d33d6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044687 Sep 19 16:32:06.276573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 736/1522] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:06 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:07.289101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:07.291299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:07.297207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:07.297684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:07.298118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:07.298849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:07.299299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:07.299722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:07.329540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6571d60a-23c0-4493-b9eb-d6f6ec466ad5 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045424 Sep 19 16:32:07.330031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 737/1523] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:07 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:07.793707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:32:07.794319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:32:07 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:32:08.341294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:08.343457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:08.349767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:08.350243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:08.350710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:08.351394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:08.351841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:08.352262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:08.382115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2083b6c2-e053-4d70-9d2d-2d1dfe037763 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044860 Sep 19 16:32:08.382700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 738/1524] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:08 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:09.394826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:09.397448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:09.404000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:09.404481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:09.404923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:09.405616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:09.406068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:09.406526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:09.423576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:09.437162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-62950c78-4812-4b4d-817c-7cdf700a0a67 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046400 Sep 19 16:32:09.437718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 739/1525] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:09 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:10.449730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:10.451886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:10.458235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:10.458789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:10.459222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:10.459900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:10.460343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:10.460759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:10.489096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fef6114a-2123-4bbf-aab5-557e6d5a1815 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043315 Sep 19 16:32:10.490135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 740/1526] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:10 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:11.502104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:11.504282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:11.511252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:11.511729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:11.512166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:11.512852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:11.513304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:11.513726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:11.543155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69d5c044-0a6b-44ab-80ab-043b254ba55d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045054 Sep 19 16:32:11.543757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 741/1527] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:11 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:12.555914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:12.558057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:12.564753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:12.565526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:12.565963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:12.566681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:12.567138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:12.567561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:12.597125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-defabb83-79bb-4b8a-8dfd-c1d0c3ef3346 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045136 Sep 19 16:32:12.597685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 742/1528] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:13.610762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:13.612994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:13.620297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:13.621252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:13.621972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:13.623060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:13.623523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:13.623956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:13.655420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8ca553b1-eb6a-48f5-ab3f-0ab814bafdef tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048668 Sep 19 16:32:13.655984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 743/1529] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:13 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:14.668016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:14.670177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:14.676784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:14.677291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:14.677725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:14.678923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:14.679384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:14.679855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:14.709239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ff2d799a-d982-44d3-8a50-004e79dcf5a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045178 Sep 19 16:32:14.709787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 744/1530] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:14 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:15.720944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:15.723112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:15.729575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:15.730053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:15.730524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:15.731219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:15.731667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:15.732093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:15.761634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ed3f6c4d-6e3d-4ec7-9ec3-d7ea4d5cee2b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044634 Sep 19 16:32:15.762206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 745/1531] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:16.774079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:16.776924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:16.783418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:16.783896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:16.784328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:16.785016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:16.785464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:16.785885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:16.815346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eafe5f90-bca2-415c-ad41-528eba22e6b8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045225 Sep 19 16:32:16.815901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 746/1532] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:32:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:32:17.766924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:32:17.767409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:32:17.773368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:17.773841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:17.774270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:17.775473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:32:17.775923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:32:17.776340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:32:27.771644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=57 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:27.794773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=48 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:32:35.981622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:32:35.982253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:32:35 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:32:37.487987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:32:37.488530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:32:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:33:10.938468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:33:11.441891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:33:17.986887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:33:17.989066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:33:17.990582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:33:17.991634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:33:17.993630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:17.998051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:33:17.999354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) Sep 19 16:33:18.000729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.001997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-7288e1c6-b6bf-43eb-ad6f-816029d95c1f) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:33:18.001997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-23d2b384-21ac-4970-9a1c-5b7c0c0d4375 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 120.244899 Sep 19 16:33:18.001997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 787/1533] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:31:17 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 120246 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:33:18.001997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:18.002352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:18.007596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.008087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.008535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.009241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.009693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.010114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.027823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:33:18.030747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:33:18.032144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:33:18.033022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:33:18.036601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.039405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:33:18.042624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) Sep 19 16:33:18.045158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:33:18.047961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ff008fc4-3975-4aaf-a331-0df9dfa2a6bd) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:33:18.047961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9388005-9d68-4596-9776-d1bff5317a56 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 60.269355 Sep 19 16:33:18.047961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 747/1534] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:32:17 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 60271 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:33:18.048453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d5b652f4-deb7-40d3-9d4f-4155499dc7bb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051970 Sep 19 16:33:18.048453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 788/1535] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:18.049901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:18.050958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:18.056214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:18.056537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:18.062935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.063428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.063867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.064579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.065030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.065447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.071401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.071993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.072552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.073345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.073856 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.074350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.096402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:33:18.197741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9de6e6d7-b4dc-4d38-b850-2a5d350aca57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 204 len: 0 microversion: 2.20 time: 0.162739 Sep 19 16:33:18.198513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 748/1536] 10.222.0.57 () {60 vars in 1280 bytes} [Mon Sep 19 16:33:18 2022] DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 19 16:33:18.206361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:18.208707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:18.214972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.215470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.215913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.216619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.217072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.217501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.248762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f5e0878f-0612-476c-afe8-e4b0e868cd81 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048795 Sep 19 16:33:18.249317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 749/1537] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:18 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:18.253967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:18.254605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:18.260200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.260678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.261115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.261801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:18.262251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:18.262703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:18.403654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:18.412495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:18.413947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:18.415159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3a8e117e-1b9b-4443-b768-636f568b059a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1803 microversion: 2.20 time: 0.164931 Sep 19 16:33:18.415706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 750/1538] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1803 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:33:19.261388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:19.263656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:19.270655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:19.271138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:19.271573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:19.272274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:19.272724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:19.273149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:19.304289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c8dd471e-9f62-42fc-b3c9-37ddbb591862 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046935 Sep 19 16:33:19.304987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 751/1539] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:19.442057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:19.442778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:19.449368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:19.449881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:19.450327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:19.451092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:19.451581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:19.452007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:19.516760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:19.529654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:19.531125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:19.532312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5117482f-83bf-4794-b850-f6793a0229ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1735 microversion: 2.20 time: 0.094477 Sep 19 16:33:19.532886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 752/1540] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1735 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:19.759402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5", "name": "network-vif-deleted", "tag": "a106b202-dd25-4a35-83f5-ddb4ba4b8d10"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:33:19.772024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:19.772607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:19.773296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:19.786965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] Creating event network-vif-deleted:a106b202-dd25-4a35-83f5-ddb4ba4b8d10 for instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 on hv-cinder-85837 Sep 19 16:33:19.793599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-f9b7d393-0150-48fa-b666-2738f0c7b055 req-2dee6f26-478d-4053-9acd-5e604e90dfd4 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037703 Sep 19 16:33:19.794663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 753/1541] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:33:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:20.318231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:20.320600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:20.327371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:20.327848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:20.328276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:20.328958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:20.329399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:20.329815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:20.361063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5f0656e9-5915-46bb-9318-cee540840160 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047564 Sep 19 16:33:20.361617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 754/1542] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:20.560109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:20.561848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:20.569649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:20.570277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:20.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:20.571769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:20.572377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:20.572911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:20.644948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:20.654063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:20.655533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:20.656688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ce92125c-2e6d-42f4-908a-596083c90c8d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.101694 Sep 19 16:33:20.657242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 755/1543] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:21.373705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:21.375920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:21.382258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:21.382779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:21.383319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:21.383902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:21.384357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:21.384780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:21.414158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3a7cf274-0812-40c9-998b-cb924f2052b1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044401 Sep 19 16:33:21.414789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 756/1544] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:21 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:21.681359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:21.681972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:21.688021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:21.688502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:21.688934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:21.689628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:21.690078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:21.690535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:21.752750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:21.761670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:21.763194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:21.764806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aa1bbeea-f16a-4142-abae-9d18ee4a78b7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086886 Sep 19 16:33:21.765365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 757/1545] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:22.427618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:22.429916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:22.437025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:22.437506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:22.437951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:22.438689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:22.439141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:22.439733 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:22.470527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-078ac026-392c-4838-b7d5-e07ec3401b41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047143 Sep 19 16:33:22.471089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 758/1546] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:22 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:22.790714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:22.791322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:22.797939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:22.798436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:22.798868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:22.799561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:22.800012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:22.800427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:22.866777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:22.875596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:22.877039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:22.878188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cf4ff7be-f433-4a6e-815a-ccb8c2a064ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.091478 Sep 19 16:33:22.878852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 759/1547] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:22 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:23.483152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:23.485494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:23.492446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:23.492931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:23.493373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:23.494071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:23.494563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:23.494988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:23.526643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38e093e3-d097-4700-abc2-43498cd97005 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047345 Sep 19 16:33:23.527228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 760/1548] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:23 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:23.903626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:23.904232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:23.913036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:23.913508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:23.913935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:23.914665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:23.915112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:23.915524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:23.983563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:23.992363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:23.993823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:23.995077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0da67d60-d08a-4e5f-880a-bc1b8f2aac2b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.095429 Sep 19 16:33:23.995638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 761/1549] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:23 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:24.539284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:24.541660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:24.548277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:24.548756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:24.549190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:24.549888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:24.550335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:24.550791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:24.577967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:24.583199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c138fd22-407e-4f9d-95e5-1f5e26fcca17 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047963 Sep 19 16:33:24.583767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 762/1550] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:24 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:25.021422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:25.022030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:25.029808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:25.030292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:25.030897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:25.031611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:25.032066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:25.032500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:25.099202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:25.108167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:25.109617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:25.110784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f1d8cfd5-5555-4302-bd73-72b00c30e1a3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.093270 Sep 19 16:33:25.111345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 763/1551] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:25 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:25.596719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:25.599137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:25.605907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:25.606426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:25.606867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:25.607580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:25.608600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:25.609023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:25.643265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-764bd834-64e7-4006-85c8-923ed69f5329 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050614 Sep 19 16:33:25.643919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 764/1552] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:25 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:26.136709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:26.137309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:26.144294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:26.144823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:26.145289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:26.145987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:26.146529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:26.147015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:26.212018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:26.221751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:26.223265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:26.224453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-622d7fa2-e812-4f80-8242-7a392f76609a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.091663 Sep 19 16:33:26.224983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 765/1553] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:26 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:33:26.656283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:26.658465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:26.666362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:26.666876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:26.667311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:26.668009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:26.668461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:26.668890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:26.701547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-41f8a977-17e9-4811-b63f-859d0bb5d034 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049673 Sep 19 16:33:26.702116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 766/1554] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:26 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:27.251736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:27.252363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:27.259047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:27.259548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:27.259985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:27.260682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:27.261140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:27.261568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:27.325987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:27.335442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:27.336863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:27.337995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dd48e148-c93c-4487-baf2-74f9fc3cc70e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.090708 Sep 19 16:33:27.338578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 767/1555] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:27 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:27.714079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:27.716239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:27.722673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:27.723153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:27.723597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:27.724286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:27.724736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:27.725159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:27.755463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-789a5576-b6cb-4b6c-ad5a-7d124deabd6d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045370 Sep 19 16:33:27.756012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 768/1556] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:27 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:28.062347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:28.096359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=14 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:28.363417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:28.364058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:28.370052 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:28.370575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:28.371019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:28.371704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:28.372158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:28.372579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:28.433333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:28.442013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:28.443490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:28.444629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-24c3c27d-d9a0-47fd-a651-514caaaf2785 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085709 Sep 19 16:33:28.445187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 769/1557] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:28 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:28.768493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:28.770700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:28.777299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:28.777789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:28.778228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:28.778975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:28.779697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:28.779864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:28.809827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cd2a8d12-2998-4580-bd61-26972f814626 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045831 Sep 19 16:33:28.810410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 770/1558] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:28 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:29.470440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:29.471050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:29.477123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:29.477596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:29.478024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:29.478728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:29.479179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:29.479593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:29.539615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:29.548731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:29.550169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:29.551329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-be5a843a-b902-439a-bbcc-c0ab89324aa1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085313 Sep 19 16:33:29.552139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 771/1559] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:29 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:29.823078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:29.825284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:29.831896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:29.832379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:29.832816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:29.833514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:29.833965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:29.834410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:29.864108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3647cc8-44f9-48c1-ae0e-63583d422770 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045012 Sep 19 16:33:29.864673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 772/1560] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:29 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:30.576211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:30.576811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:30.583308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:30.583785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:30.584215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:30.584900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:30.585351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:30.585771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:30.645933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:30.654403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:30.655840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:30.656966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69315d16-ed35-4906-a41c-586091c6961e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.084629 Sep 19 16:33:30.657530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 773/1561] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:30 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:30.877049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:30.879131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:30.885314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:30.885803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:30.886243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:30.887014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:30.887472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:30.887899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:30.917562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-893064b1-0543-45ad-a837-3a12b5b5b64d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044634 Sep 19 16:33:30.918116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 774/1562] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:30 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:31.682944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:31.683542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:31.690398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:31.690874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:31.691293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:31.691970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:31.692410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:31.692824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:31.756851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:31.766265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:31.767753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:31.768897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aba742ce-f33d-4d81-9bca-4ced03e6bd7e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.089950 Sep 19 16:33:31.769452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 775/1563] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:31 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:31.930511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:31.932670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:31.939155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:31.939571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:31.940001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:31.940681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:31.941127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:31.941548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:31.973945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2910bdcd-624c-48ce-8d5a-df795f84cd6f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047297 Sep 19 16:33:31.974923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 776/1564] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:31 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:32.793714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:32.794324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:32.800809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:32.801285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:32.801972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:32.802680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:32.803132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:32.803555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:32.864146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:32.872597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:32.874087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:32.875262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9a954868-71b7-48c6-909e-f0e850dfb777 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085530 Sep 19 16:33:32.875816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 777/1565] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:32 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:32.987415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:32.989859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:32.996203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:32.996678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:32.997108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:32.997791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:32.998237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:32.998691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:33.027949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-68332f96-be18-4190-a483-ad5cd0470b7d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044952 Sep 19 16:33:33.028508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 778/1566] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:32 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:33.900995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:33.901607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:33.908572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:33.909054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:33.909509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:33.910233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:33.910692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:33.911118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:33.929879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:33.977381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:33.986789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:33.988217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:33.989339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5e8c402e-23ea-42ad-a99e-df6461bc853d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.092370 Sep 19 16:33:33.989898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 779/1567] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:33 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:34.041243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:34.043286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:34.049727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:34.050206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:34.050661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:34.051348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:34.052055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:34.052473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:34.080810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9cb15337-7b62-41b6-b3b3-3d6ed763eb28 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043469 Sep 19 16:33:34.081361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 780/1568] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:34 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:33:35.013180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:35.013851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:35.020284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:35.020750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:35.021177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:35.021857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:35.022299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:35.022746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:35.084029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:35.092838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:35.094272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:35.095424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0007b462-7432-4ed2-9414-ef8d4c3ee3bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086138 Sep 19 16:33:35.095972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 781/1569] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:35 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:35.100452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:35.102568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:35.108291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:35.109287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:35.109717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:35.110408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:35.110855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:35.111270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:35.139954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-01e1db00-a92a-40f8-bcfb-0130a64d3b12 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042994 Sep 19 16:33:35.140510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 782/1570] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:35 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:36.120803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:36.121430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:36.128443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:36.130410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:36.130410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:36.130410 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:36.130702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:36.130927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:36.197628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:36.206790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:36.208211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:36.209330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6f9821b1-0dfd-4f4c-af97-f3f1a0cb3c89 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.092594 Sep 19 16:33:36.209875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 783/1571] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:36 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:36.214422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:36.216516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:36.222567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:36.223047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:36.223478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:36.224165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:36.224605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:36.225021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:36.256498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b11fe260-104d-4338-8438-bf8a6f93ea0a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045620 Sep 19 16:33:36.257042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 784/1572] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:36 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:37.235891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:37.236558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:37.242941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:37.243418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:37.243849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:37.244536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:37.244978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:37.245397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:37.307260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:37.315840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:37.317274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:37.318440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6680a558-b46a-49b0-bd55-b2baedb44d4f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087442 Sep 19 16:33:37.318991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 785/1573] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:37 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:37.323425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:37.325534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:37.331700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:37.332177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:37.332609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:37.333288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:37.333732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:37.334155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:37.363135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ee43a841-ad8f-46b1-bb1b-fdeb4fc5cdfa tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043148 Sep 19 16:33:37.363687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 786/1574] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:37 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:38.345280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:38.346398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:38.358671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:38.359560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:38.360369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:38.361615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:38.362057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:38.362849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:38.424825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:38.433819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:38.435806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:38.436991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eff227d4-b20e-4134-9198-b467a56d6da4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.096733 Sep 19 16:33:38.437535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 787/1575] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:38 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:38.441976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:38.444094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:38.450196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:38.450708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:38.451794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:38.452476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:38.452977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:38.453399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:38.482927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5a5b48de-1ecd-407b-97bf-ec5de7353a5e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044231 Sep 19 16:33:38.483452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 788/1576] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:38 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:39.462479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:39.463111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:39.469969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:39.470465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:39.470908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:39.471588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:39.472033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:39.472451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:39.534745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:39.543575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:39.545034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:39.546182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f1096714-b5d0-4cab-b3b9-2209f68bd522 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087627 Sep 19 16:33:39.546771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 789/1577] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:39 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:39.551370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:39.553743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:39.559638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:39.560124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:39.560676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:39.561368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:39.561819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:39.562270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:39.590932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-46ca14b3-b7e1-4c1c-9ede-00c3dd635050 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043175 Sep 19 16:33:39.591477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 790/1578] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:39 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:40.571601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:40.572277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:40.578885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:40.579366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:40.579810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:40.580538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:40.580989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:40.581416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:40.648792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:40.658236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:40.659717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:40.660870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f07644f0-ff74-455a-be1c-c85955895f44 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.093400 Sep 19 16:33:40.661432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 791/1579] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:40 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:40.665987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:40.668136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:40.674657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:40.675133 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:40.675572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:40.676270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:40.676994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:40.677411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:40.708879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-16d650f9-eb6c-4302-b244-8d80a86f64fb tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.046390 Sep 19 16:33:40.709448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 792/1580] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:40 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:41.688778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:41.689381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:41.696519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:41.697001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:41.697436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:41.698130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:41.698618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:41.699045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:41.762178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:41.772079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:41.773563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:41.774737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-867c902c-2fad-4969-92fc-7943bc79d51f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.091797 Sep 19 16:33:41.775301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 793/1581] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:41 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:41.779932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:41.782004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:41.784848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=99,nova_api:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:41.788611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:41.789091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:41.789529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:41.790267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:41.790702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:41.791128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:41.822184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-90bfee11-c308-4cd9-b370-58af59cee68c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045857 Sep 19 16:33:41.822770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 794/1582] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:41 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:42.800142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:42.800752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:42.807477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:42.807949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:42.808380 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:42.809072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:42.809514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:42.809939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:42.870756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:42.879810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:42.881246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:42.882382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c77550c6-6c70-4493-bdb7-2c7532eb2b37 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086174 Sep 19 16:33:42.882947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 795/1583] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:42 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:42.887390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:42.889480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:42.895745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:42.896220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:42.896655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:42.897339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:42.897784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:42.898208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:42.901878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:42.927161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-739a0173-f585-4b77-83ae-0ab10d2a8a71 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042918 Sep 19 16:33:42.927713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 796/1584] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:42 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:43.908340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:43.908982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:43.915403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:43.915880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:43.916315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:43.917003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:43.917453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:43.917878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:43.981467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:43.990145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:43.991625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:43.992772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a09b78ec-e285-43e6-92d8-33df0af7665f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.088913 Sep 19 16:33:43.993324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 797/1585] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:43 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:43.997774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:44.000180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:44.008804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:44.009136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:44.009579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:44.010284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:44.010769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:44.011194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:44.042485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6bf88a4b-cda0-45db-85ee-8d9d1ef263b2 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047974 Sep 19 16:33:44.042986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 798/1586] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:43 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:45.021337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:45.024448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:45.029130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:45.029554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:45.029985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:45.030724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:45.031176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:45.031596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:45.098791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:45.107361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:45.108811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:45.110592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e31222b8-c309-44c9-9056-80e9d65d4490 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.093263 Sep 19 16:33:45.111142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 799/1587] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:45 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:45.115659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:45.117750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:45.123345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:45.123818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:45.124227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:45.124927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:45.125369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:45.125782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:45.156223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3421f0d8-8f16-4d37-99ff-bda97e20d26b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043594 Sep 19 16:33:45.156770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 800/1588] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:45 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:46.135919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:46.136545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:46.143402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:46.143885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:46.144331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:46.145027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:46.145477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:46.145900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:46.208502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:46.217025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:46.218498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:46.219633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cd639ba7-198d-49ca-811c-4df14fbcf594 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.088054 Sep 19 16:33:46.220188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 801/1589] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:46 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:46.224616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:46.226735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:46.232284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:46.232759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:46.233220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:46.234419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:46.234873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:46.235293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:46.264384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cc7957e8-56b5-430f-999f-b450daec1956 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043205 Sep 19 16:33:46.264934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 802/1590] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:46 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:47.245559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:47.246236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:47.252855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:47.253331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:47.253758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:47.254482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:47.254900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:47.255318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:47.316478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:47.325139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:47.326624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:47.328370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-61721271-8ac2-434f-a195-511860d33ba2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086157 Sep 19 16:33:47.328911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 803/1591] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:47 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:47.338227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:47.338227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:47.341511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:47.341983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:47.342442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:47.343122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:47.343569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:47.343984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:47.371833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2dad7f17-fa4f-4d66-8ad9-bf25835ae661 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.041944 Sep 19 16:33:47.372381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 804/1592] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:47 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:48.353225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:48.353923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:48.361156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:48.361540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:48.361981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:48.362705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:48.363157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:48.363583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:48.423950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:48.433175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:48.434629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:48.435779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9c4be0aa-5267-48c7-92cb-691d0b434b50 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086535 Sep 19 16:33:48.436330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 805/1593] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:48 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:48.445642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:48.445642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:48.448926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:48.449409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:48.449845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:48.450575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:48.451026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:48.451445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:48.481265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ae5a4cce-a2f7-4108-a681-85277ab99858 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043893 Sep 19 16:33:48.481811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 806/1594] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:48 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:49.461549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:49.462233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:49.468572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:49.469050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:49.469484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:49.470206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:49.470688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:49.471112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:49.534216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:49.542722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:49.544170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:49.545305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-afe91b6c-a483-4bdf-8c62-76afc8c8ba63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087638 Sep 19 16:33:49.545860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 807/1595] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:49 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:49.550860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:49.553231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:49.558945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:49.559427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:49.559865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:49.560552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:49.561006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:49.561432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:49.592018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a09a33cc-c869-4665-b3b9-40ef318a57f6 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044634 Sep 19 16:33:49.592571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 808/1596] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:49 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:50.571673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:50.572378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:50.578699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:50.579175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:50.579608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:50.580290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:50.580738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:50.581158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:50.641698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:50.650385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:50.652134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:50.653275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d48331cd-7a24-4058-8147-cd23e390a53b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085572 Sep 19 16:33:50.653845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 809/1597] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:50 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:50.658815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:50.660924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:50.666713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:50.667191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:50.667628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:50.668670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:50.669127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:50.669550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:50.698776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-579c5c4f-fc60-4620-b787-2833842156f9 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043916 Sep 19 16:33:50.699329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 810/1598] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:50 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:51.291643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:33:51.292219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:33:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:33:51.679285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:51.679892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:51.686177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:51.686680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:51.687714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:51.688399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:51.688844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:51.689264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:51.752373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:51.761208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:51.762701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:51.763841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-982a5ff0-03f0-45cd-804a-b3a635297fe2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.088789 Sep 19 16:33:51.764396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 811/1599] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:51 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:51.768931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:51.771061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:51.777685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:51.777915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:51.778347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:51.779071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:51.779516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:51.779937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:51.805594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:33:51.810645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-10d6cdae-ce9c-4a0e-9da2-061a254c31ec tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045220 Sep 19 16:33:51.811204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 812/1600] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:51 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:52.788120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:52.788728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:52.790951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:33:52.791088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:33:52 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (ip 10.222.0.57) !!! Sep 19 16:33:52.798244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:52.798753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:52.799179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:52.799858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:52.800295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:52.800711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:52.865709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:52.874815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:52.876275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:52.877409 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-efcd3e22-be5e-4839-8e6f-0387c0759bec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.093200 Sep 19 16:33:52.877963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 813/1601] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:52 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:52.882603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:52.884741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:52.891211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:52.891769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:52.892202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:52.892922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:52.893371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:52.893795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:52.922778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dd1bf354-dcc2-4b92-92ba-cc5167114ba1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043757 Sep 19 16:33:52.923327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 814/1602] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:52 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:53.903339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:53.903990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:53.910855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:53.911332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:53.911765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:53.912490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:53.912950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:53.913369 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:53.978518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:53.987294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:53.988675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:53.989821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8c00075c-6ad0-4393-bb4f-a7fdb0fc8ff7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.090392 Sep 19 16:33:53.990404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 815/1603] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:53 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:53.994981 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:53.997093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:54.002875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:54.003367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:54.003807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:54.004500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:54.004955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:54.005381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:54.044692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-27a33677-eb63-468a-951f-f8de7ffaefaf tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.053187 Sep 19 16:33:54.045484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 816/1604] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:53 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:55.015275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:55.015920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:55.022407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:55.023255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:55.024014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:55.025219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:55.025997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:55.026770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:55.098703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:55.107259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:55.109358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:55.110502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ed73ea52-fedb-4473-bea4-8e7acea77ce8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.099195 Sep 19 16:33:55.111057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 817/1605] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:55 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:55.115545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:55.117622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:55.123193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:55.123663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:55.124100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:55.125113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:55.125559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:55.125977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:55.154357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-efcb0dc0-eef6-4c05-86bd-0bbc4477674f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042306 Sep 19 16:33:55.154927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 818/1606] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:55 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:56.135533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:56.136188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:56.142631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:56.143114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:56.143553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:56.144252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:56.144705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:56.145132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:56.215398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:56.224079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:56.225525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:56.226695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-979d0947-75be-4800-9534-5b56fc3b4f9c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.095145 Sep 19 16:33:56.227254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 819/1607] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:56 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:56.231720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:56.234334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:56.240131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:56.240612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:56.241048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:56.241735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:56.242188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:56.242641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:56.271764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-903185b1-1a43-405d-b329-9a50336d212b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043493 Sep 19 16:33:56.272323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 820/1608] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:56 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:57.252763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:57.253425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:57.259815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:57.260293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:57.260729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:57.261686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:57.262170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:57.262614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:57.323942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:57.333155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:57.334631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:57.335768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-122abcc2-7216-4ed3-8862-e871dae807a2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087421 Sep 19 16:33:57.336349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 821/1609] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:57 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:57.345630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:57.345630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:57.348838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:57.349317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:57.349754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:57.350467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:57.350921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:57.351349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:57.380011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-729b43d3-773a-4848-86e1-06b1349d2058 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042677 Sep 19 16:33:57.380563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 822/1610] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:57 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:58.361684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:58.362454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:58.368777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:58.369255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:58.369690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:58.370391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:58.370852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:58.371275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:58.439154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:58.447781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:58.449211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:58.450341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b3432839-2f7d-4c7c-94f5-782febb0430a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.093059 Sep 19 16:33:58.450934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 823/1611] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:58 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:58.455668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:58.457793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:58.463668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:58.464141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:58.464573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:58.465256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:58.465702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:58.466199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:58.496765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-217b79b3-a0db-4634-80ea-321e4bab2406 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044828 Sep 19 16:33:58.497300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 824/1612] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:58 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:33:59.475567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:59.476318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:59.482142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:59.482636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:59.483065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:59.483737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:59.484176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:59.484587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:59.544954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:59.553728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:59.555169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:33:59.556289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d0706e40-5811-47e2-af3a-0e39c3d0f9ac tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.084700 Sep 19 16:33:59.556842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 825/1613] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:33:59 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:33:59.561255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:33:59.563847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:33:59.569335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:59.569795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:59.570221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:59.570927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:33:59.571368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:33:59.571780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:33:59.601155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9860bf5c-5283-484a-b3a1-08b4f19ae720 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043323 Sep 19 16:33:59.601702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 826/1614] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:33:59 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:00.582777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:00.583452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:00.589320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:00.589792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:00.590226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:00.590949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:00.591405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:00.591832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:00.652387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:00.661582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:00.663055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:00.664224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-534cc810-5e7b-4004-b51d-1f1f3dafc5f3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085905 Sep 19 16:34:00.664809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 827/1615] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:00 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:00.669340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:00.671480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:00.677730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:00.678220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:00.678713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:00.679428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:00.679881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:00.680309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:00.709833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c6eba2de-0d81-40f9-87a0-481655cd47e7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044015 Sep 19 16:34:00.710417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 828/1616] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:00 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:01.689566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:01.690271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:01.696646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:01.697128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:01.697565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:01.698249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:01.698721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:01.699147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:01.716920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:01.761263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:01.770008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:01.771475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:01.772593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dac64bb7-1695-4d62-8fd7-b3b948acf545 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087038 Sep 19 16:34:01.773152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 829/1617] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:01.777900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:01.779959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:01.792528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:01.792999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:01.793424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:01.794114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:01.794588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:01.795002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:01.832841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3e81a493-1faf-4b64-8b29-43b48bf1c94e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.058649 Sep 19 16:34:01.833401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 830/1618] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:01 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:02.801149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:02.802049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:02.808465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:02.808940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:02.809376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:02.810067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:02.810558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:02.810985 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:02.874852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:02.883376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:02.884818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:02.885943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3c540172-7260-458f-b980-4daa2ccd0719 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.088767 Sep 19 16:34:02.886521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 831/1619] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:02 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:02.891505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:02.893608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:02.899280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:02.899757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:02.900188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:02.900871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:02.901513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:02.901939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:02.931143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0a086920-f24b-4127-b6fd-3f00b8ce2600 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043621 Sep 19 16:34:02.931695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 832/1620] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:02 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:03.911691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:03.912374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:03.918747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:03.919220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:03.919655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:03.920351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:03.921434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:03.921856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:03.983238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:03.992364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:03.993789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:03.995012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7a50fed7-5bf3-4ea6-baa6-bfdb0ae557f0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087219 Sep 19 16:34:03.995567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 833/1621] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:03 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:03.999979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:04.002045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:04.007577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:04.008051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:04.008478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:04.009157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:04.009602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:04.010017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:04.040476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-42991ed2-527d-4dd8-ae4a-8329f0ddd9c8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043810 Sep 19 16:34:04.040953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 834/1622] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:03 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:05.020107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:05.020775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:05.027322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:05.027804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:05.028246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:05.028988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:05.029388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:05.029811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:05.091260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:05.099818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:05.101256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:05.102428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2ef1b145-ae8c-4009-825c-f2f390c93339 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.086304 Sep 19 16:34:05.102974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 835/1623] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:05 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:05.107510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:05.110098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:05.115765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:05.116247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:05.116685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:05.117366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:05.117834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:05.118242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:05.147009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0dc19dcd-f13f-478f-b891-1b89553ae18b tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043031 Sep 19 16:34:05.148295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 836/1624] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:05 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:06.127998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:06.128614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:06.134818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:06.135285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:06.135711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:06.136386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:06.136826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:06.137241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:06.203868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:06.212507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:06.213953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:06.215120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d531babe-dcf4-43e7-a259-5ac00ad60aa2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.091111 Sep 19 16:34:06.215675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 837/1625] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:06 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:06.221295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:06.223424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:06.229420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:06.230131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:06.230608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:06.231334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:06.231786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:06.232213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:06.261613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7bdd6987-cf0a-4f02-9025-55529739f215 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044797 Sep 19 16:34:06.262360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 838/1626] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:06 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:07.240426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:07.241065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:07.247868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:07.248351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:07.249406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:07.250158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:07.250666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:07.251134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:07.315050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:07.323612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:07.325068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:07.326207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9f85b1b-4662-4201-94a7-779ac7f3a02b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.089718 Sep 19 16:34:07.326821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 839/1627] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:07 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:07.331326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:07.333489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:07.339205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:07.339678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:07.340119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:07.340792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:07.341244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:07.341667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:07.371709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aade466b-decf-4103-ae46-b6689d254422 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043872 Sep 19 16:34:07.372304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 840/1628] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:07 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:08.351079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:08.351725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:08.358352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:08.359042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:08.359485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:08.360177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:08.360633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:08.361057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:08.425083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:08.433920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:08.435366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:08.436997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-be34935d-38b1-4460-ad61-621791cf2cb4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.089318 Sep 19 16:34:08.437543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 841/1629] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:08 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:08.442065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:08.444186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:08.449854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:08.450328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:08.450891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:08.451657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:08.452716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:08.453145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:08.482591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f1739acc-5223-4a86-b0ee-0a386e9ada3d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043996 Sep 19 16:34:08.483165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 842/1630] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:08 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:09.466598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:09.467229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:09.474067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:09.474593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:09.475021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:09.475700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:09.476146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:09.476567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:09.536886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:09.545414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:09.546871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:09.547988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-147ebb1a-da98-46e6-b980-2ed5636d43b6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085328 Sep 19 16:34:09.548532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 843/1631] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:09 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:09.553248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:09.555345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:09.557932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:09.561891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:09.562358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:09.562811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:09.563485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:09.563930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:09.564345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:09.593457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a8c5482f-93a2-43f9-9747-69ef1ec65aab tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043345 Sep 19 16:34:09.593999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 844/1632] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:09 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:10.772331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:10.772935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:10.779622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:10.779845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:10.780472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:10.783137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:10.783619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:10.784053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:10.846159 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:10.854654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:10.856096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:10.857247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e834fd25-278e-4aab-ad05-c5b16781bca9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.288441 Sep 19 16:34:10.857801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 845/1633] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:10 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:10.976944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:10.979217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:10.985619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:10.986097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:10.986575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:10.987267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:10.987721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:10.988147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:10.991784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:11.017770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-767cfb7a-cc29-4db3-8698-8a025360beb8 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.158942 Sep 19 16:34:11.018314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 846/1634] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:10 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:11.883595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:11.884258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:11.891280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:11.891764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:11.892201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:11.892900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:11.893350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:11.893776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:11.957710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:11.966284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:11.968411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:11.969550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d19f6806-e37d-4a99-83f6-50ca4332440a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.089910 Sep 19 16:34:11.970109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 847/1635] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:11 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:12.029823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:12.032600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:12.038599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:12.039081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:12.039518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:12.040206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:12.040657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:12.041079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:12.071515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-87d7e7c2-b66f-442c-8807-5e11476365c1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.045439 Sep 19 16:34:12.072084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 848/1636] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 47 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:34:12.994365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:12.995060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:13.001866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:13.002344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:13.002810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:13.003512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:13.003961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:13.004392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:13.314948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:13.323865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:13.325288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:13.326428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8e1b550b-d422-409b-be1a-094625ef39e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.335968 Sep 19 16:34:13.327500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 849/1637] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:13.331936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:13.334001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:13.339709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:13.340182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:13.340613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:13.341290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:13.341735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:13.342150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:13.370707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-886007ac-5647-4f61-81b6-9906feee3e9e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042216 Sep 19 16:34:13.371248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 850/1638] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:13 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:14.351691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:14.352376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:14.359517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:14.359995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:14.360435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:14.361124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:14.361575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:14.362000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:14.424063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:14.432932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:14.434412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:14.435549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3e5fdf2a-07b9-4db7-bb20-a2735ae9996a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087739 Sep 19 16:34:14.436105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 851/1639] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:14 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:14.440551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:14.442663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:14.448172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:14.448651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:14.449089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:14.449776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:14.450228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:14.450681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:14.479224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-780302a7-4b25-4b90-a335-c5f7e54debde tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.042129 Sep 19 16:34:14.479773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 852/1640] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:14 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:15.460256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:15.460909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:15.467175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:15.467640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:15.468067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:15.468741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:15.469186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:15.469599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:15.531726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:15.540811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:15.542249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:15.543423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b3c253e7-a9db-424f-8ce2-10b444b62653 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.087025 Sep 19 16:34:15.543978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 853/1641] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:15 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:15.549004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:15.551135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:15.557190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:15.557663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:15.558100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:15.558810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:15.559260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:15.559886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:15.588978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-25355c2d-b68a-43c2-bffe-9e4532004812 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043993 Sep 19 16:34:15.589530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 854/1642] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:15 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:16.569932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:16.570671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:16.577522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:16.577996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:16.578458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:16.579145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:16.579589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:16.580007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:16.640548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:16.649015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:16.650486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:16.651624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3813067d-eff4-498e-b629-e109acd5f39a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085592 Sep 19 16:34:16.652177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 855/1643] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:16 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 87 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:16.656630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:16.658749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:16.664295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:16.664770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:16.665205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:16.665890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:16.666338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:16.667028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:16.696387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-98906136-9592-445f-a3fd-f7149502a29f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043219 Sep 19 16:34:16.696939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 856/1644] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:16 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:17.677819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:17.678525 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:17.685018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.685490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.685915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:17.686642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.687085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.687503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:17.752117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:17.760704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:17.762132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:34:17.763285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eb8398f6-d2cc-430f-a7e8-868cc4826033 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.089638 Sep 19 16:34:17.763831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 857/1645] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:34:17 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:34:17.768920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:17.771031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:17.776827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.777301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.777729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:17.778427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.778870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.780474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:17.808820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f67b82ab-570a-4eda-ac56-85a5ea1939a4 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.043473 Sep 19 16:34:17.809375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 858/1646] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:34:17 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:34:17.977510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:34:17.978089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:34:17.985326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.985780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.986286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:17.987155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:34:17.987660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:34:17.988229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:34:27.982912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=30 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:34:28.006773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=80 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:35:11.847982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:35:18.297927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:35:21.694737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:35:21.695430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:35:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:35:21.853441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:35:27.809620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:35:27.810185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:35:27 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:35:28.302208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:35:55.991788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:35:55.992438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:35:55 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:35:57.684462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:35:57.685014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:35:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:37:11.301938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:37:11.302508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:37:11 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:37:12.126794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:37:12.129148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:37:12.131031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:37:12.131989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:37:12.133151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:12.134424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:37:12.136135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) Sep 19 16:37:12.137690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:37:12.139772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-c814dcd2-1c99-4f18-a96f-9cba634188a7) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:37:12.139772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9244ccaf-f948-4bdb-ac36-c8598baaeff7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 234.084801 Sep 19 16:37:12.139772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 789/1647] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:33:18 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 234086 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:37:12.139772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:12.140748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:12.144305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.144791 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.145230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.148332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.148332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.148332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.219345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:12.230317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:12.231868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:12.233167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4242fdc3-c39e-4f68-a951-1376205aaa21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.100992 Sep 19 16:37:12.233749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 790/1648] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:37:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:37:12.238904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:12.241077 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:12.247095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.247568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.247997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.248684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.249127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.249546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.279472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ac6e855f-d616-45a2-ad4a-7c976041464e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.044570 Sep 19 16:37:12.280031 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 791/1649] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:37:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 46 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:37:12.284635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:12.285114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:12.290562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.291033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.291461 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.292785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:12.293230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:12.293646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:12.823945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:37:12.824517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:37:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:37:18.564048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:37:18.566435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:37:18.567835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:37:18.568513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:37:18.569600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:18.570870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:37:18.572422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) Sep 19 16:37:18.574142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:37:18.578475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-e02700a6-f8a7-4a1f-963d-07978aaf1041) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:37:18.578475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b56f8065-0e47-4b68-bcb0-59edc9c4f7f2 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 180.690712 Sep 19 16:37:18.578475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 859/1650] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:34:17 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 269 bytes in 180692 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:37:18.578475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:18.578875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:18.593101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.593913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.594504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.595308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.595830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.596413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.622469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:37:18.711674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cfb3648a-0159-44f8-bb50-7ae21497be4c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.141290 Sep 19 16:37:18.712437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 860/1651] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:37:18 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:37:18.718569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a3e0db80-db85-4971-97e8-a31473ce6749 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc3118430>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:18.748243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a3e0db80-db85-4971-97e8-a31473ce6749 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 202 len: 0 microversion: 2.1 time: 0.034669 Sep 19 16:37:18.749006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 861/1652] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:37:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 0 bytes in 36 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:37:18.754960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:18.755691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:18.763537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.764020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.764452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.765161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.765612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.766174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.833142 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:18.842795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:18.844238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:37:18.845390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b4c40334-6b4a-4c26-ab9f-6bf33b5dfcc0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.095127 Sep 19 16:37:18.846477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 862/1653] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:37:18 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:37:18.850803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:18.852904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:18.859080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.859559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.860064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.860777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.861195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.861544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.894542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4d7eeb18-49a9-409c-a4ae-9da55d75136d tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047375 Sep 19 16:37:18.895246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 863/1654] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:37:18 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 49 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:37:18.900028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:37:18.900520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:37:18.907134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.907590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.908017 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:18.908743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:37:18.909186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:37:18.909607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:37:22.289030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:22.312007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=16 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:28.904413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:37:28.930746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=30,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:38:47.820907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:38:47.821406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:38:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:39:12.244214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:39:16.004891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:39:16.005584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:39:16 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:39:19.046255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:39:22.255625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=4 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:39:29.059859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:40:21.708365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:40:21.709029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:40:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:40:31.313305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:40:31.313935 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:40:31 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:40:32.833321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:40:32.833805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:40:32 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:41:12.417515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:41:12.420241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:41:12.421709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:41:12.422458 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:41:12.423866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.425107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:12.426205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) Sep 19 16:41:12.427512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-355aabfc-748d-460c-8d04-289954906ff9) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 240.140572 Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:41:12 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:41:12 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (10.222.0.57) Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: CRITICAL nova [None req-ca5405fe-0c61-4b3d-8c09-6635878ec8ba tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Unhandled error: OSError: write error Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova OSError: write error Sep 19 16:41:12.429247 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova Sep 19 16:41:12.429864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 792/1655] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:37:12 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 240142 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:41:12.513244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:12.514536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:12.520568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.521044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.521474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.522156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.522627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.523042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.548929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:41:12.596181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.compute.api [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:41:12.597010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-01ddb128-7f19-44ed-8ee9-bfc4b296a1c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.173242 Sep 19 16:41:12.597569 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 793/1656] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:41:12 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:41:12.602168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c5a4c78f-51ae-4680-b905-5c1732eab14e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc3159700>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:12.628840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-c5a4c78f-51ae-4680-b905-5c1732eab14e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c5a4c78f-51ae-4680-b905-5c1732eab14e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:41:12.632548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.634298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:41:12.635220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:41:12.636813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:12.638475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:41:12.640000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c5a4c78f-51ae-4680-b905-5c1732eab14e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:41:12.640000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c5a4c78f-51ae-4680-b905-5c1732eab14e tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.035305 Sep 19 16:41:12.640000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 794/1657] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:41:12 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:41:12.640000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:12.640407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:12.645810 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.646279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.646740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.647417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.647861 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.648277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.710542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:12.720429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:12.722259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:12.723450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1aa7e7d0-b898-4530-b1f1-0a78f4a21bab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.088132 Sep 19 16:41:12.724544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 795/1658] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:41:12 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:41:12.779745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:12.781888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:12.787895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.788378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.788820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.789545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.790006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.790463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.822438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-36f8b9c0-b9ee-4c3a-abd5-6a4c609b9545 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.096703 Sep 19 16:41:12.823024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 796/1659] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:41:12 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:41:12.827877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:12.828429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:12.834135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.834641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.835070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:12.835755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:12.836203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:12.836619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.200528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:41:19.202952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:41:19.204685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:41:19.205762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:41:19.208058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.211422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.212758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) Sep 19 16:41:19.214128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-b9a41c0f-98e2-406c-adc2-8ede562e90cf) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 240.308542 Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:41:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:41:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (10.222.0.57) Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: CRITICAL nova [None req-a74e8f13-9a6d-440a-bb91-5453bcef802f tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Unhandled error: OSError: write error Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova OSError: write error Sep 19 16:41:19.215468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova Sep 19 16:41:19.216082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 864/1660] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:37:18 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 240311 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:41:19.216082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.216082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:19.220199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.221350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.221878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.222971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.223435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.223862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.252735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:41:19.303208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:41:19.304314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-927fc8ef-961a-4fdc-87aa-511127320ba0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.096751 Sep 19 16:41:19.304907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 865/1661] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:41:19 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:41:19.309479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65f943ae-5c68-4d14-b951-a500cb49eaf5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc31590d0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.326200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-65f943ae-5c68-4d14-b951-a500cb49eaf5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65f943ae-5c68-4d14-b951-a500cb49eaf5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:41:19.330107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.331892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:41:19.332693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:41:19.333904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:41:19.335157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.336342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65f943ae-5c68-4d14-b951-a500cb49eaf5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:41:19.336342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-65f943ae-5c68-4d14-b951-a500cb49eaf5 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.025555 Sep 19 16:41:19.336342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 866/1662] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:41:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 27 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:41:19.336592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.336934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:19.344244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.344740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.345160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.345839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.346287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.346827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.360189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:41:19.361735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:41:19.363326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:41:19.364443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:41:19.365997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:41:19.367642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:41:19.367642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5612323b-3386-4675-b8ea-6ff0ca28e8d2 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.030192 Sep 19 16:41:19.367642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 867/1663] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:41:19 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 31 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:41:19.367946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.368548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:19.374028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.374538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.374971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.375648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.376094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.376519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.437776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:19.446939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:19.448390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:41:19.449572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2af5c257-c2a9-44c5-b21b-8d03d461720f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.085065 Sep 19 16:41:19.450128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 868/1664] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:41:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:41:19.454743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.456869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:19.465891 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.466902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.468513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.468513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.468513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.468950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.498565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cde3979b-5565-458d-87fe-d9554aeff1f7 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047348 Sep 19 16:41:19.499112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 869/1665] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:41:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:41:19.503543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:41:19.504055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:41:19.509384 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.509858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.510290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:19.511004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:41:19.511450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:41:19.511869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:41:22.832170 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=10 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:41:22.856764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=29,nova_cell1:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:41:29.507109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=12 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:41:29.530897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=31,nova_cell1:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:42:36.014925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:42:36.015554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:42:36 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:42:37.701571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:42:37.702181 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:42:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:43:12.570579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:43:12.843721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:43:19.355054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:43:19.357235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:43:19.359621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:43:19.362363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:43:19.365350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.368305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.371029 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) Sep 19 16:43:19.375579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-71bfe666-df55-40cf-84cc-c11721aa5944) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 126.534708 Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (10.222.0.57) Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: CRITICAL nova [None req-d2def6b0-f17c-4c75-aa6e-62967150b464 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Unhandled error: OSError: write error Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova OSError: write error Sep 19 16:43:19.379111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova Sep 19 16:43:19.380549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 797/1666] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:41:12 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 126536 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.380549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.380549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.380549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.381305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.381305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.381305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.381305 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.382927 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.399563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:19.402351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:43:19.403848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:43:19.404528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:43:19.405638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.406953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.409071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) Sep 19 16:43:19.411515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.413747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-ab6ce6ab-c27d-43f0-9ca0-7e7f407c8e93) {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 119.905135 Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (10.222.0.57) Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: CRITICAL nova [None req-f660f4a6-4f75-4e6f-82b2-b0b354190f6b tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Unhandled error: OSError: write error Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova OSError: write error Sep 19 16:43:19.413901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova Sep 19 16:43:19.414877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 870/1667] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:41:19 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 119908 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.418755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc31625e0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.438240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:19.441405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.443476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:19.444303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:19.445506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.459016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.034386 Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 (ip 10.222.0.57) !!! Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 (10.222.0.57) Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: CRITICAL nova [None req-73592e5d-34d5-486a-890c-13408d471ba3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Unhandled error: OSError: write error Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova OSError: write error Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova Sep 19 16:43:19.461785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 871/1668] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 0 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.478539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.compute.api [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:19.479356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.118346 Sep 19 16:43:19.479878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:43:19.479878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - uwsgi_response_write_headers_do(): Broken pipe [core/writer.c line 248] during DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (10.222.0.57) Sep 19 16:43:19.480450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: CRITICAL nova [None req-f77d1024-dfb4-4467-abee-313b064d10ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Unhandled error: OSError: write error Sep 19 16:43:19.480450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova OSError: write error Sep 19 16:43:19.480450 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova Sep 19 16:43:19.480664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 798/1669] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.489360 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.490116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.508727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.509232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.509680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.510411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.510870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.511295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.522540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.523038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.531364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.531854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.532536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.533229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.533686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.534118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.552389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:19.553970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.556029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.557015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.558601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.110197 Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 (ip 10.222.0.57) !!! Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 (10.222.0.57) Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: CRITICAL nova [None req-0b549c2a-c928-4593-bba2-f5dd5cc28a57 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Unhandled error: OSError: write error Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova OSError: write error Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova Sep 19 16:43:19.560226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 872/1670] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 0 bytes in 112 msecs (HTTP/1.1 404) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.561913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.564991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.572206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.572711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.573330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.574630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.575387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.575822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.591146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.600556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.602205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.604113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.122067 Sep 19 16:43:19.604654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (ip 10.222.0.57) !!! Sep 19 16:43:19.604654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (10.222.0.57) Sep 19 16:43:19.605256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: CRITICAL nova [None req-fda97b55-57be-4f3f-bdb0-02764a4d2773 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Unhandled error: OSError: write error Sep 19 16:43:19.605256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova OSError: write error Sep 19 16:43:19.605256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova Sep 19 16:43:19.605611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 799/1671] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 0 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.609876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.051306 Sep 19 16:43:19.611021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments (ip 10.222.0.57) !!! Sep 19 16:43:19.611021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:43:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments (10.222.0.57) Sep 19 16:43:19.611656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: CRITICAL nova [None req-f9e1b8e7-2dfc-4461-a2f6-852cc0027230 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Unhandled error: OSError: write error Sep 19 16:43:19.611656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova OSError: write error Sep 19 16:43:19.611656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova Sep 19 16:43:19.611953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 873/1672] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 0 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:43:19.612435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.613035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.618877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.619550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.621501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.622054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.622576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.623610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.624276 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.624885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.626543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.627036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.627462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.628361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.628829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.629230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.656554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:19.713756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:19.714690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f3cf5102-8e25-4fe9-bd41-44f2d45741ac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.101990 Sep 19 16:43:19.716226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 874/1673] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:43:19.721041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97a2e954-0fe3-4c7e-8487-c687cf92dff3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc30a40d0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.736763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-97a2e954-0fe3-4c7e-8487-c687cf92dff3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97a2e954-0fe3-4c7e-8487-c687cf92dff3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:19.739880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.741755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:19.742714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:19.744123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:19.745632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.747018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97a2e954-0fe3-4c7e-8487-c687cf92dff3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.747018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-97a2e954-0fe3-4c7e-8487-c687cf92dff3 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.024732 Sep 19 16:43:19.747018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 875/1674] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 26 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:19.747018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.747018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.759183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.759703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.760143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.760866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.761316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.761757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.780752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:19.782902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.784605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:19.785569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:19.787082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:19.788771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:19.788771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-57e21a58-5673-45e4-b91e-fed085477da6 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.042111 Sep 19 16:43:19.788771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 876/1675] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:19.791500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.792095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.799332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.799812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.800242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.801233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.801374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.801769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.873968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.883512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.884993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:19.886173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-78d8e675-6405-4414-9c02-9a0ac585e6dd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.098587 Sep 19 16:43:19.886788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 877/1676] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:19 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:19.891577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.893777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.899990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.900484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.900921 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.901640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.902095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.902571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.935023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-40feb9ec-b01d-4908-97eb-51a9c765a851 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.047158 Sep 19 16:43:19.935608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 878/1677] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:19 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:19.940244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:19.940731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:19.947255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.947732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.948163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.948860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:19.949302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:19.949712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:19.977697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:20.023908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:20.024721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-520f76ec-3de2-464f-b423-878dde58f434 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.088212 Sep 19 16:43:20.025290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 879/1678] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:43:20.029792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58430877-e01c-48e0-8e99-a78b1dce0799 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc317ed30>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.052070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-58430877-e01c-48e0-8e99-a78b1dce0799 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58430877-e01c-48e0-8e99-a78b1dce0799 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.056076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.057822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:20.058817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:20.060513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.062207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.063520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58430877-e01c-48e0-8e99-a78b1dce0799 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.063520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58430877-e01c-48e0-8e99-a78b1dce0799 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.032246 Sep 19 16:43:20.063520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 880/1679] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.070542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.071622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.082098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.082861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.083415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.084302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.084809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.085325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.101655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.103342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.106688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.108418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.113210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-20f581b2-ba75-445c-8920-628614a71645 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.043369 Sep 19 16:43:20.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 881/1680] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.115437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.118701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.119188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.119645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.120342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.120795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.121222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.202032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.218342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.220888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.222050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-866d80a3-dfa1-487d-bc1f-9965992725e6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.115453 Sep 19 16:43:20.223631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 882/1681] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 117 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:20.228930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.242067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.242771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.243327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.244084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.244633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.245235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.293295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7eb56c9b-ab90-46e4-bdf2-f876d115822c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.068566 Sep 19 16:43:20.293967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 883/1682] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:20.299458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.300311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.309221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.309715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.310153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.310976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.311350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.312663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.345243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:20.397609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:20.398458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-37e6dfd8-1f63-4216-8985-890c7fb12560 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.103538 Sep 19 16:43:20.399334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 884/1683] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 105 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:43:20.403667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5571bdd3-61f4-4358-947c-31ed629a57a7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadbf62caf0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.417263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-5571bdd3-61f4-4358-947c-31ed629a57a7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5571bdd3-61f4-4358-947c-31ed629a57a7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.419185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.421394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:20.422504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:20.423951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.425377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.426806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5571bdd3-61f4-4358-947c-31ed629a57a7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.426806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5571bdd3-61f4-4358-947c-31ed629a57a7 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.020501 Sep 19 16:43:20.426806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 885/1684] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.426806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.427126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.433919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.434451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.434885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.435576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.436020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.436512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.454861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.457493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.459083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.460507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.462285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.464125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.464125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9e587c44-37ad-4c70-b498-591aa3c7b37d tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.038086 Sep 19 16:43:20.464125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 886/1685] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.471307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.472475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.482743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.483754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.484640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.485870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.486823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.487596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.569816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.582455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.589660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.590849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-79f21c01-85de-424a-ba8b-992bbb07d4bb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.127348 Sep 19 16:43:20.591405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 887/1686] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:20.598871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.602004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.611012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.611835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.612579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.613757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.614692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.615851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.652375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-28a077b1-1770-4077-8698-90b267d21a6e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.059735 Sep 19 16:43:20.652969 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 888/1687] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:20.657837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.658330 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.664455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.664930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.665359 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.666056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.666570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.667121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.686136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:20.695704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:20.748549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:20.749532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-77dc0e24-2e70-4749-8817-5bec73f15338 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.095503 Sep 19 16:43:20.750920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 889/1688] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 97 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:43:20.755960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-050377dd-d815-45c4-aa87-67a5f751c07f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc31058b0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.768848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-050377dd-d815-45c4-aa87-67a5f751c07f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-050377dd-d815-45c4-aa87-67a5f751c07f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.771066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.772756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:20.773519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:20.775270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:20.776760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.778350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-050377dd-d815-45c4-aa87-67a5f751c07f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.778350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-050377dd-d815-45c4-aa87-67a5f751c07f tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.020698 Sep 19 16:43:20.778350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 890/1689] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 22 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.778350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.783388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.787045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.787537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.787999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.788821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.789306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.789758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.805579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:20.807201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.809067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:20.810151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:20.811892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:20.814499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:20.814499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e65d13ab-73b7-4325-aee2-68394a42e459 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.034511 Sep 19 16:43:20.814499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 891/1690] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:20.814499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.815235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.821707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.822188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.822661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.823356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.823875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.824485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.898220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.907514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.908988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:20.910179 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5b775682-eee3-40b9-a587-ffcf668a04f7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.100121 Sep 19 16:43:20.911108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 892/1691] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:20.916035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.918236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.924598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.925091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.925531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.926245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.926735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.927170 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.960421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-81f4bbaf-462d-4da5-947a-5341c71ff87a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048200 Sep 19 16:43:20.961313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 893/1692] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:20 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:20.966018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:20.966550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:20.975550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.976044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.976485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:20.977191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:20.977645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:20.978074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.006363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:43:21.059549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.compute.api [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Instance is already in deleting state, ignoring this request Sep 19 16:43:21.060383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a0098d6-70a0-46c5-83ff-f6fcc87f1281 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 204 len: 0 microversion: 2.1 time: 0.098144 Sep 19 16:43:21.060952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 894/1693] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:20 2022] DELETE /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:43:21.066207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad3cbd9a-45a6-46a9-a800-2ab656941bd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Calling method '.version_select at 0x7fadc3105a60>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.078299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-ad3cbd9a-45a6-46a9-a800-2ab656941bd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] HTTP exception thrown: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad3cbd9a-45a6-46a9-a800-2ab656941bd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Request method failure captured: Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 HTTP/1.1 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJWZ_jutK33Ax4GQkNrTiDs_5ppxI46xWK5xEngkAYaCHJKPyvTdLvaVoAthw0Rzar8YEyZR1HsrdbpHhPMQ4H9D8flHPvCzFdg30kKzskN2eyD-IipnISCApoPEPTJwaJobrB_elaEbfwnf3iip9-0_rVPfoqycTsRjNSt4VVA1eJs Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:21.080004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/2a726afa4ec84ec99a5b1bd08c95a0cd"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_2a726afa4ec84ec99a5b1bd08c95a0cd"}]}] Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 2a726afa4ec84ec99a5b1bd08c95a0cd Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestVolumeBootPattern-357900851 Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestVolumeBootPattern-357900851-project Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.081472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'tempest-TestVolumeBootPattern-1392065781'} Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 170, in _delete Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.api.delete_key_pair(context, user_id, id) Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 71, in wrapped Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _emit_versioned_exception_notification( Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.force_reraise() Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise self.value Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/exception_wrapper.py", line 63, in wrapped Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(self, context, *args, **kw) Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6861, in delete_key_pair Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi keypair = self.get_key_pair(context, user_id, key_name) Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 6882, in get_key_pair Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return objects.KeyPair.get_by_name(context, user_id, key_name) Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 19 16:43:21.082806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 144, in get_by_name Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi db_keypair = cls._get_from_db(context, user_id, name) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 126, in _get_from_db Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return _get_from_db(context, user_id, name=name) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1010, in wrapper Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return fn(*args, **kwargs) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/keypair.py", line 40, in _get_from_db Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.KeypairNotFound(user_id=user_id, name=name) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.KeypairNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:43:21.084007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 143, in delete Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req, id) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/keypairs.py", line 172, in _delete Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=exc.format_message()) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 Sep 19 16:43:21.086210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad3cbd9a-45a6-46a9-a800-2ab656941bd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] Returning 404 to user: Keypair tempest-TestVolumeBootPattern-1392065781 not found for user 9fd6c22a61be43658ae87052b867e384 {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:21.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ad3cbd9a-45a6-46a9-a800-2ab656941bd0 tempest-TestVolumeBootPattern-357900851 tempest-TestVolumeBootPattern-357900851-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781" status: 404 len: 146 microversion: 2.1 time: 0.019503 Sep 19 16:43:21.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 895/1694] 10.222.0.57 () {58 vars in 1263 bytes} [Mon Sep 19 16:43:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1392065781 => generated 146 bytes in 21 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:21.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.088739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:21.094101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.094616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.095056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.095746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.096190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.096711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.110627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:21.112374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.114096 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:21.114962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.119967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.122231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:21.122231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c7c58792-9e16-48c9-a9f6-ccaf9d918ec5 tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.030627 Sep 19 16:43:21.122231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 896/1695] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:43:21 2022] DELETE /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 32 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:21.122231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.122231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:21.126583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.127066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.127734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.128223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.128683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.129023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.206960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:21.217043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:21.221464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:21.229522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-47351464-2227-4ee9-87c4-70a8ffb74126 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 200 len: 1445 microversion: 2.20 time: 0.108161 Sep 19 16:43:21.230414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 897/1696] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:21 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 1445 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:21.246955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.246955 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:21.253495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.254553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.255445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.256823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.257796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.258676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.325702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-294372d7-4511-46a7-91ab-e9d1109ff349 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.093359 Sep 19 16:43:21.326980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 898/1697] 10.222.0.57 () {58 vars in 1302 bytes} [Mon Sep 19 16:43:21 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/os-volume_attachments => generated 197 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:21.342855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.344652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:21.355300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.356148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.356936 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.358203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.359074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.360518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.536434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:21.544234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-771b1aa4-175e-4a9f-86ca-62a47f68dbeb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.208446 Sep 19 16:43:21.545083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 899/1698] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:21 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:21.565542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.565997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: 4af93002-f79c-48e0-9a49-a348edf588c8] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:21.578694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.579412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.580888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.580888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:21.581450 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:21.581945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:21.600458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 HTTP/1.1 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:43:21.602045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.603939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4af93002-f79c-48e0-9a49-a348edf588c8'} Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:43:21.605176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. Sep 19 16:43:21.607116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:43:21.608505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance 4af93002-f79c-48e0-9a49-a348edf588c8 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:43:21.608505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-55b1eabd-c4a1-4ecf-b284-936f840f55ab tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8" status: 404 len: 111 microversion: 2.1 time: 0.055271 Sep 19 16:43:21.609957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 900/1699] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:21 2022] GET /compute/v2.1/servers/4af93002-f79c-48e0-9a49-a348edf588c8 => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:43:21.615825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-88ba9751-eb1c-4e77-865c-7dbc114caf68 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '.version_select at 0x7fadc30e7790>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:21.638250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-88ba9751-eb1c-4e77-865c-7dbc114caf68 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1741274330" status: 204 len: 0 microversion: 2.20 time: 0.026644 Sep 19 16:43:21.639014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 901/1700] 10.222.0.57 () {60 vars in 1262 bytes} [Mon Sep 19 16:43:21 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1741274330 => generated 0 bytes in 28 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 19 16:43:22.344071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Action: 'action', calling method: >, body: {"os-getConsoleOutput": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:22.348034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:22.361042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:22.361544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:22.362028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:22.362766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:22.363396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:22.364027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:22.642748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b856d70f-eee9-479d-a033-79608a034990 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action" status: 200 len: 76366 microversion: 2.1 time: 0.303002 Sep 19 16:43:22.647283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 902/1701] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:43:22 2022] POST /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2/action => generated 76366 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:43:22.659200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:22.660404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:22.675684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:22.676339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:22.676852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:22.677789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:22.678266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:22.678762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:22.949664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:22.968742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-556b4741-2038-4e25-81f8-2a73ef93529f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.314237 Sep 19 16:43:22.970536 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 903/1702] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:22.979541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fee17bd7-4600-4fe1-814d-63edd5b4d6d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: .version_select at 0x7fadc3099ee0>, body: {"keypair": {"name": "tempest-keypair-1898280496", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMTMO/RC7SvfIKNAKAIDFAmaGvAfHfhvTs3vf05qpSZFazJXoT59IWJFx59mRSzO7licOayggUvrT1p4pDSWZbdJmgy1mGsjCet5DofljM+q6URpsg+48OZNsHiymUKZfQ=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:23.017686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-fee17bd7-4600-4fe1-814d-63edd5b4d6d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:43:23.022491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-fee17bd7-4600-4fe1-814d-63edd5b4d6d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for user bd190d8e252342ca9232f7a0c966c715 and project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:43:23.048310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fee17bd7-4600-4fe1-814d-63edd5b4d6d1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.075804 Sep 19 16:43:23.049396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 904/1703] 10.222.0.57 () {62 vars in 1200 bytes} [Mon Sep 19 16:43:22 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 78 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 19 16:43:24.019959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:24.022342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:24.035415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:24.036202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:24.037820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:24.038613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:24.039868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:24.041246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:24.516632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:24.549722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bc12fc26-5d56-4611-8067-b803d1453651 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.537851 Sep 19 16:43:24.551139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 905/1704] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 540 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:25.545915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-992093263", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "668f712a-f576-4558-aa5d-ef858ef28f5d"}], "security_groups": [{"name": "tempest-securitygroup--1670023129"}], "key_name": "tempest-keypair-1898280496", "user_data": "}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:25.806719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] validate_networks() for [('668f712a-f576-4558-aa5d-ef858ef28f5d', None, None, None, None, None)] {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:43:26.797675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:43:26.799056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:43:26.800363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:43:26.801109 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:43:26.802808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:43:26.816193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:26.817063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:26.817734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:26.818860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:26.819603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:26.820287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:26.855373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:43:26.862044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Getting quotas for user bd190d8e252342ca9232f7a0c966c715 and project 344bf851ff7d471fba47bde9b6f7e219. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:43:26.882228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:43:26.886243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:43:26.924796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=95,nova_api:DELETE=1,nova_api:INSERT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:26.956099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-366d2fcd-f3d9-4a7d-bb02-a193cc9ee22c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 1.414402 Sep 19 16:43:26.956995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 906/1705] 10.222.0.57 () {62 vars in 1188 bytes} [Mon Sep 19 16:43:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 1416 msecs (HTTP/1.1 202) 10 headers in 453 bytes (2 switches on core 0) Sep 19 16:43:26.971368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:26.972892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:26.996631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:26.997880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:26.998984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:27.001105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:27.001977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:27.002678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:27.456458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:27.467646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-acc83ae3-99f4-4deb-8484-c260c8c44e77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.508576 Sep 19 16:43:27.485467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 907/1706] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 527 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:27.492609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:27.493503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:27.583091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:27.602258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:27.603949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:27.605108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fbac6d39-1901-46e5-a3be-75ca0527809a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1281 microversion: 2.20 time: 0.117836 Sep 19 16:43:27.608045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 908/1707] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:27 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1281 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:28.492499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:28.493314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:28.502246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:28.503131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:28.503743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:28.504839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:28.505633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:28.506349 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:28.766939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:28.786137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-74a5350f-cd84-4b9a-9b84-88ea68be0da4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.297837 Sep 19 16:43:28.787040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 909/1708] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:28.809567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:28.810396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:28.824509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:28.824928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:28.825371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:28.826600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:28.836556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:28.837129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:28.924101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:28.934481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:28.936284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:28.937578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1530c566-b45c-4802-8752-9d9291e2665e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1339 microversion: 2.20 time: 0.136723 Sep 19 16:43:28.938245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 910/1709] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:28 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1339 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:29.620178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:29.650479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=27,nova_cell1:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:29.822223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:29.823999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:29.838862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:29.839585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:29.841030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:29.841933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:29.842795 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:29.844048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:29.894735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:30.117819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:30.129501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b0219d9f-5a0b-4635-81ee-4733742bc37b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.317793 Sep 19 16:43:30.130797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 911/1710] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:30.136864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:30.137758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:30.145961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:30.146793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:30.147422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:30.148318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:30.148941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:30.150228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:30.282154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:30.297006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:30.300175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:30.301664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0371073a-b059-4b26-bdbd-e20ba9e70cab tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1339 microversion: 2.20 time: 0.169468 Sep 19 16:43:30.303817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 912/1711] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:30 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1339 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:31.153466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:31.154150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:31.162839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:31.163372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:31.163820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:31.164568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:31.164939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:31.165365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:31.351968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:31.363820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-717bf7ca-d3f3-43a9-b66a-e1b4d359677a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.214233 Sep 19 16:43:31.364679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 913/1712] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:31 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:31.369428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:31.370221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:31.376356 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:31.377158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:31.377828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:31.378878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:31.379552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:31.380255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:31.544800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:31.553943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:31.555805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:31.556769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38f72fce-25a0-4635-93fd-5c978fe22763 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.191081 Sep 19 16:43:31.557323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 914/1713] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:31 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:32.533243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:32.545334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:32.546127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:32.547286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:32.560756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] Creating event network-changed:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:43:32.568201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-6423495b-7a5f-4bf5-b20d-01a247a151e0 req-58ca1083-39f6-4472-9baf-6cd3397c4645 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.515362 Sep 19 16:43:32.569828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 915/1714] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:43:32 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:43:32.581093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:32.582173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:32.590603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:32.591576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:32.592189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:32.594174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:32.595064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:32.595660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:32.898269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:32.907448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d4f5fe2e-9a7f-4d23-b5e3-3943cdde95fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.335141 Sep 19 16:43:32.908124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 916/1715] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:32 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:32.927329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:32.927950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:32.934337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:32.934878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:32.935317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:32.936026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:32.936483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:32.936907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:33.108835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:33.117517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:33.119136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:33.120358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a2ef791a-8e6d-40aa-a189-054c2f862391 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.211132 Sep 19 16:43:33.121120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 917/1716] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:32 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:33.932460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:33.933140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:33.940471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:33.941227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:33.941858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:33.942620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:33.943078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:33.943527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:34.116613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:34.128653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fcb32f76-da3f-4d69-86df-2095ee3a0919 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.200162 Sep 19 16:43:34.129526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 918/1717] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:43:34.152113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:34.153151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:34.159370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:34.160091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:34.160527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:34.161375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:34.161831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:34.162245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:34.323694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:34.332984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:34.334468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:34.335590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-43527c0b-6649-44e0-810e-a2bcd758ca9e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.189407 Sep 19 16:43:34.336152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 919/1718] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:34 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:35.152495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:35.153146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:35.159557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:35.160074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:35.160509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:35.161202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:35.161653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:35.162080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:35.292441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:35.301223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b643e8e1-2390-4a97-a438-06f912cd097b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152700 Sep 19 16:43:35.301786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 920/1719] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:35.361272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:35.361876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:35.367986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:35.368466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:35.368894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:35.369582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:35.370035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:35.370488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:35.498162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:35.506823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:35.508265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:35.509376 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ad1913dc-0352-4721-92e4-2e36c1f9ea76 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.152364 Sep 19 16:43:35.509929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 921/1720] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:35 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:43:36.331483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:36.332194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:36.345390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:36.345959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:36.346423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:36.347131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:36.347581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:36.348005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:36.478559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:36.487055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-23fd972c-93a4-4ab5-88d9-dcbb6d4bd3ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160358 Sep 19 16:43:36.487764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 922/1721] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:36 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:36.533219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:36.533828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:36.539723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:36.540211 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:36.540643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:36.541334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:36.541788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:36.542212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:36.675944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:36.685064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:36.686483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:36.688099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-33e76908-e609-496c-99c6-dbb1b45e2244 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.158890 Sep 19 16:43:36.688659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 923/1722] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:36 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:43:37.511305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:37.511996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:37.513937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:37.520388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:37.520865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:37.521297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:37.521999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:37.522481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:37.522903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:37.598494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=12 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:37.652739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:37.661722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f636b22d-557b-4f1d-9753-72c5de559760 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154275 Sep 19 16:43:37.662303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 924/1723] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:37.713206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:37.713817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:37.720468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:37.720946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:37.721405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:37.722089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:37.722578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:37.723007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:37.860729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:37.869464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:37.870919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:37.872040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-05211ab9-8812-48f4-9db0-01c6e05a5a43 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.162867 Sep 19 16:43:37.872606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 925/1724] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:37 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:43:38.367996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "name": "network-vif-plugged", "status": "completed", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:38.378655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:38.379135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:38.379568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:38.393027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] Creating event network-vif-plugged:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:43:38.398051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-bbbbdd91-825d-4ef7-a0d9-090d28152f16 req-0be34c3a-3372-4291-9aee-eae6848fea7e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.226373 Sep 19 16:43:38.398939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 926/1725] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:43:38 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:43:38.685902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:38.686587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:38.693573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:38.694064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:38.694538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:38.695254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:38.695882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:38.696674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:38.832310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:38.840777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-63c3a0ac-be79-4429-b3d9-8830706a6e6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159176 Sep 19 16:43:38.841353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 927/1726] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:38 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:38.903049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:38.903671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:38.910278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:38.910794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:38.911235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:38.911940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:38.912435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:38.912861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:39.045899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:39.054355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:39.056087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:39.057202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ab653bb3-8f99-4c1e-bc89-639c08c26430 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1407 microversion: 2.20 time: 0.158265 Sep 19 16:43:39.057761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 928/1727] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:38 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1407 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:43:39.866341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:39.867015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:39.873262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:39.873750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:39.874184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:39.875526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:39.875982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:39.876416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:39.897510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:40.024070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:40.032702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e9086fb1-be32-4c73-8e63-fd1e28e477d0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.171170 Sep 19 16:43:40.033280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 929/1728] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:39 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:40.085732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:40.087438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:40.102345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:40.103292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:40.104108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:40.105390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:40.106238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:40.106768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:40.253713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:40.261792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:40.263264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:40.264416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4141c1e3-c298-4889-8b90-38461a99ce77 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1599 microversion: 2.20 time: 0.186512 Sep 19 16:43:40.264984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 930/1729] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:43:40 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1599 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:43:40.289527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:40.290031 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:40.295792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:40.296919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:40.297364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:40.298053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:40.298535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:40.298963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:40.380300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cea90df6-bcfe-4a7a-b95a-de17e39d80c0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.094509 Sep 19 16:43:40.380903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 931/1730] 10.222.0.57 () {60 vars in 1316 bytes} [Mon Sep 19 16:43:40 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-interface => generated 284 bytes in 96 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 19 16:43:41.060594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:41.061298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:41.068534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:41.069019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:41.069497 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:41.070200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:41.070689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:41.071119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:41.287893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:41.297044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3a4b9631-72fd-4ddd-bfc2-14a967ff8b16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.240341 Sep 19 16:43:41.297638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 932/1731] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:41.710804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:43:41.721440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:41.722064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:41.722508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:41.734919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] Creating event network-changed:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:43:41.740726 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-efc569d7-7206-4a02-8f34-5c65bf7927cb req-dd55b4a3-44fb-4d89-9009-9785d9696c69 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.093676 Sep 19 16:43:41.741506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 933/1732] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:43:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:43:42.321800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:42.322594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:42.329721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:42.330206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:42.330682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:42.331385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:42.331840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:42.332266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:42.473305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:42.481933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-02373adc-2f12-4903-b1a9-95d760688397 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.164310 Sep 19 16:43:42.482814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 934/1733] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:43.505883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:43.506633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:43.513168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:43.513637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:43.514095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:43.514807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:43.515778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:43.516198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:43.653111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:43.662078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f9fa3663-0b38-4260-9c0b-80daec9f1123 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160033 Sep 19 16:43:43.662700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 935/1734] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:43 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:44.686116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:44.686835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:44.693264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:44.693747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:44.694184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:44.694926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:44.695378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:44.695807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:44.832615 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:44.840285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-039e2582-b3bf-499b-b20e-c9b9352d6409 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158400 Sep 19 16:43:44.840876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 936/1735] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:44 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:45.864670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:45.865315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:45.871610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:45.872084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:45.872518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:45.873199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:45.873654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:45.874071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:46.011321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:46.019634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5475cbfd-5583-473e-b6a3-914d10927bbb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158926 Sep 19 16:43:46.020204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 937/1736] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:45 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:47.043938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:47.044732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:47.051337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:47.051817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:47.052257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:47.052945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:47.053394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:47.053816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:47.183268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:47.191873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2ceb5aac-a18f-4b58-9751-ad12a32b5b85 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151963 Sep 19 16:43:47.192436 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 938/1737] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:47.833984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:43:47.834713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:47 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:43:48.215529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:48.216138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:48.223255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:48.223725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:48.224153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:48.224826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:48.225278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:48.225697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:48.352262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:48.360812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d4f6cfb6-e541-4fdb-9171-fc2b3e7d5fe0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149139 Sep 19 16:43:48.361385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 939/1738] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:49.384476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:49.385182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:49.391487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:49.391950 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:49.392379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:49.393057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:49.393498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:49.393911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:49.525403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:49.534245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-198818f7-1f17-4851-96db-724a6dc55558 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153620 Sep 19 16:43:49.534838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 940/1739] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:50.563657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:50.564392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:50.571185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:50.571665 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:50.572089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:50.572787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:50.573259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:50.573676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:50.704131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:50.712404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3e4d50d9-cda7-492b-903c-a736ab266cf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153145 Sep 19 16:43:50.712998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 941/1740] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:50 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:51.322598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:43:51.323177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:43:51 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:43:51.736474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:51.737068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:51.743556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:51.744024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:51.744452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:51.745137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:51.745580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:51.745996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:51.885253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:51.893796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3d8e63f7-f7fd-4827-942b-3ab937fcb154 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161638 Sep 19 16:43:51.894427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 942/1741] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:51 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:52.918566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:52.919236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:52.927208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:52.927684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:52.928119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:52.928819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:52.929294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:52.929970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:53.064427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:53.073191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2d9045be-fca5-4ff5-a66b-05b0329aa367 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159153 Sep 19 16:43:53.073774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 943/1742] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:52 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:54.097573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:54.098217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:54.104444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:54.104933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:54.105342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:54.106039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:54.106509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:54.106925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:54.238087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:54.246364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-36dba3ee-08d9-4d3d-bc5d-418d69aabaf0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153225 Sep 19 16:43:54.246967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 944/1743] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:55.269145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:55.269845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:55.276260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:55.276739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:55.277178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:55.277864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:55.278317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:55.278768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:55.410107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:55.418351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8df31702-b147-44fa-8641-46c6ca1ae866 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153058 Sep 19 16:43:55.418949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 945/1744] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:56.443341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:56.443993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:56.450393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:56.450895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:56.451331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:56.452026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:56.452479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:56.452908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:56.584469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:56.592582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-48b6383b-0938-48e9-9111-5b57b8e9da28 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153169 Sep 19 16:43:56.593146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 946/1745] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:57.616493 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:57.617131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:57.619501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=83 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:57.625444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:57.625913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:57.626365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:57.627068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:57.627508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:57.627943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:57.632071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:43:57.762778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:57.771537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f3c487bf-eeeb-4821-8501-29ce10fd576b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158923 Sep 19 16:43:57.772113 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 947/1746] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:57 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:58.795734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:58.796348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:58.803004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:58.803483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:58.803923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:58.804876 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:58.805329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:58.805761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:58.933986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:43:58.942611 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5dfcf9ed-3015-4373-838a-79ebb4fb2613 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150765 Sep 19 16:43:58.943184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 948/1747] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:58 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:43:59.966623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:43:59.967305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:43:59.973961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:59.974484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:59.974931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:43:59.975631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:43:59.976082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:43:59.976513 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:00.105146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:00.113739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-84b36587-b02e-47a7-b965-cc4b09cef742 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151336 Sep 19 16:44:00.114309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 949/1748] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:43:59 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:01.137086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:01.137796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:01.144011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:01.144485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:01.144919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:01.145600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:01.146048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:01.146498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:01.277557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:01.288153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c73af25c-bd9f-46dd-a5dd-9aee3f9be0b4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154777 Sep 19 16:44:01.288735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 950/1749] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:02.313431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:02.314137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:02.320414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:02.320885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:02.321319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:02.321993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:02.322458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:02.322879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:02.454166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:02.462337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6695c715-c681-4d20-aa43-1a1b653584ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153434 Sep 19 16:44:02.462925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 951/1750] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:02 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:03.485083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:03.485845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:03.492340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:03.492808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:03.493239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:03.493924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:03.494387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:03.494816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:03.628116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:03.637213 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-efa8614d-caad-4273-9a74-a626ee4a763e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156075 Sep 19 16:44:03.637789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 952/1751] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:04.661561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:04.662164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:04.668605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:04.669084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:04.669518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:04.670225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:04.670708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:04.671172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:04.812908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:04.821965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d1066da6-7229-409f-9fb1-6718b0c8fdc1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.164230 Sep 19 16:44:04.822925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 953/1752] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:04 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:05.846237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:05.846916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:05.853268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:05.853742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:05.854203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:05.854944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:05.855378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:05.855799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:05.987821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:05.996091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-97b3d3ea-1c15-4f99-9279-7034c73aa71c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153806 Sep 19 16:44:05.996700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 954/1753] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:05 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:07.020907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:07.021542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:07.027822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:07.028322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:07.028759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:07.029453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:07.029903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:07.030332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:07.161348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:07.169394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aade7907-ee47-42ed-b903-33b7545d6be1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152553 Sep 19 16:44:07.169952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 955/1754] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:07 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:08.193533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:08.194210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:08.200593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:08.201071 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:08.201510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:08.202203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:08.203177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:08.203602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:08.334571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:08.342845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3d97d984-e7c4-439f-bfad-f25474a4f203 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153176 Sep 19 16:44:08.343998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 956/1755] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:08 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:09.366649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:09.367377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:09.374131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:09.374640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:09.375078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:09.375772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:09.376228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:09.376654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:09.507429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:09.515821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0f07e19f-f33e-4140-9f25-96d13bfaba52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153182 Sep 19 16:44:09.516398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 957/1756] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:09 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:10.539843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:10.540510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:10.547592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:10.548073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:10.548530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:10.549228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:10.549674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:10.550125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:10.680895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:10.695468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3452c456-e9aa-4ddb-88ec-080191091612 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159505 Sep 19 16:44:10.696082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 958/1757] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:10 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:11.719232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:11.719915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:11.726278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:11.726784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:11.727215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:11.727916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:11.728362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:11.728783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:11.867162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:11.876130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-552ec97f-5460-4b80-93ef-ed965d19fb42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160801 Sep 19 16:44:11.876710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 959/1758] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:11 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:44:12.904486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:12.906332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:12.918918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:12.919808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:12.920639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:12.922045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:12.922939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:12.923751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:13.085175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:13.100124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cd18e458-dd2a-45b4-8f74-6d434b008bc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.202893 Sep 19 16:44:13.101132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 960/1759] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:12 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:14.140489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:14.141287 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:14.148868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:14.149408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:14.149877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:14.150654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:14.151135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:14.151595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:14.289027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:14.297754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b8599368-5ea7-41f0-a117-c78b0bba2117 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161208 Sep 19 16:44:14.298400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 961/1760] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:14 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:15.320886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:15.321557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:15.327869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:15.328345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:15.328782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:15.329470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:15.329923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:15.330351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:15.465423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:15.474097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a8ff4a91-0348-4767-b6bb-f9bdd02b9124 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157137 Sep 19 16:44:15.474704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 962/1761] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:15 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:16.498865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:16.499568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:16.506582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:16.507051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:16.507477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:16.508164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:16.508603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:16.509020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:16.636677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:16.640143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:16.645767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eddbda6a-56ec-4073-9fb6-504ccefe719e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151275 Sep 19 16:44:16.646342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 963/1762] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:16 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:16.650865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:16.651353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:16.657130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:16.657602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:16.658040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:16.658755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:16.659209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:16.659638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:16.682999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4c8b7079-f33f-41a0-a67f-9635d843152a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/ips" status: 200 len: 152 microversion: 2.20 time: 0.035653 Sep 19 16:44:16.683562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 964/1763] 10.222.0.57 () {60 vars in 1289 bytes} [Mon Sep 19 16:44:16 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/ips => generated 152 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:44:17.669166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:17.669834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:17.676286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:17.676763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:17.677197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:17.677893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:17.678343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:17.678793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:17.816489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:17.824693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-439dec36-c09f-44f7-b8a5-efeb926aab2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159431 Sep 19 16:44:17.825263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 965/1764] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:18.849542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:18.850185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:18.856715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:18.857195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:18.857623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:18.858347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:18.858826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:18.859244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:18.990100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:18.998402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d7e80858-3e9f-4374-8f03-ea1fb4b34fd2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152683 Sep 19 16:44:18.998984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 966/1765] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:18 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:20.021226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:20.021930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:20.029790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:20.030845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:20.031256 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:20.031963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:20.032415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:20.032835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:20.173023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:20.181519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-484d8990-26d8-4e19-ab5f-fc1fc703dda8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.164211 Sep 19 16:44:20.182084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 967/1766] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:21.205197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:21.205838 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:21.213020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:21.213692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:21.221238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:21.221238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:21.221238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:21.221238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:21.352826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:21.362108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-54f07ad4-a350-42f0-ac59-353143e89c03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161304 Sep 19 16:44:21.362717 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 968/1767] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:21 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:22.385227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:22.385958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:22.393222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:22.393696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:22.394119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:22.394842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:22.395289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:22.395706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:22.540101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:22.549573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-492c9456-e827-4be1-8b48-0fac29185389 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.168193 Sep 19 16:44:22.550155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 969/1768] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:23.574303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:23.575036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:23.582216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:23.582731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:23.583169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:23.583862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:23.584315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:23.584747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:23.721840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:23.730209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2720f790-c05d-46ba-b3c2-e17fd5f95840 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159813 Sep 19 16:44:23.730820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 970/1769] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:23 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:24.753353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:24.754048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:24.761453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:24.761946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:24.762398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:24.763264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:24.763646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:24.764077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:24.896677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:24.905849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9b0cc931-83ba-40f6-9ba2-06bcbd8c79af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155564 Sep 19 16:44:24.906498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 971/1770] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:44:25.929373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:25.930336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:25.936869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:25.937346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:25.937788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:25.938531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:25.938987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:25.939413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:26.111755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:26.121002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f79e3a4a-0483-47e9-ac2d-c935b30ab649 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.195516 Sep 19 16:44:26.121630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 972/1771] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:25 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:27.148845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:27.149617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:27.156350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:27.156988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:27.157750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:27.159048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:27.159872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:27.160643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:27.304523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:27.313760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-631ab7cb-486f-46f7-8dc7-afb6fafa1dfd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.172689 Sep 19 16:44:27.314338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 973/1772] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:27 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:44:28.337598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:28.338250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:28.340490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=54 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:28.346025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:28.346548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:28.346974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:28.347707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:28.348156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:28.348583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:28.486212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:28.500340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f780940c-4a30-4b78-a82e-e09ea1fbaf0e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.166564 Sep 19 16:44:28.500941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 974/1773] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:29.524462 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:29.525141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:29.531967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:29.532454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:29.532896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:29.533604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:29.534057 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:29.534510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:29.672552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:29.681461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-59e3c56e-9024-4a43-9ee9-c548d1adedc7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160879 Sep 19 16:44:29.682050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 975/1774] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:30.705931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:30.706656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:30.713173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:30.713688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:30.714183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:30.714897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:30.715348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:30.715778 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:30.848362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:30.856826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1e3e282f-ea45-47e5-8b05-be9ae1e590b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155545 Sep 19 16:44:30.857400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 976/1775] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:30 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:31.880694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:31.881377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:31.888382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:31.888865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:31.889300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:31.890001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:31.890478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:31.890907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:32.036328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:32.045954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1ded566c-59ce-4bf6-8498-f97a7b9a089e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.168623 Sep 19 16:44:32.046863 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 977/1776] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:31 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:33.070265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:33.070961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:33.078069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:33.079045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:33.079484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:33.080234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:33.080715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:33.081442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:33.241522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:33.252112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-45e42737-b274-4479-8a52-6cb779f12565 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.185694 Sep 19 16:44:33.252705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 978/1777] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:34.275563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:34.276285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:34.282804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:34.283297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:34.283723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:34.284427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:34.284875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:34.285299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:34.417517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:34.426298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2843f78d-37f1-4570-9b78-41ac90e942d7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154689 Sep 19 16:44:34.426843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 979/1778] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:34 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:35.450486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:35.451167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:35.458034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:35.458537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:35.459042 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:35.459725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:35.460173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:35.460593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:35.593893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:35.597339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:35.602647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4539e797-8f95-483c-bec9-8e2aa2bcbe18 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156118 Sep 19 16:44:35.603218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 980/1779] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:36.626024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:36.626755 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:36.634201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:36.634898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:36.635177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:36.635896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:36.636352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:36.636784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:36.771467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:36.780916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d92ef89b-c80c-49fe-b232-6350d9a0bfb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158815 Sep 19 16:44:36.781478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 981/1780] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:36 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:37.805663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:37.806357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:37.820009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:37.820509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:37.820949 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:37.821666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:37.822121 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:37.822582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:37.976367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:37.985377 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-14de27f1-0ca7-4317-9ed7-b40e7a504df5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.183912 Sep 19 16:44:37.985954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 982/1781] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:39.008932 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:39.009606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:39.016209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:39.016683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:39.017112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:39.017799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:39.018244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:39.018684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:39.151670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:39.160156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8b81a890-60e4-4b32-8c55-244186b7ed16 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155152 Sep 19 16:44:39.160734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 983/1782] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:39 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:44:40.184904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:40.185590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:40.192060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:40.192511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:40.192948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:40.193650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:40.194102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:40.194566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:40.333187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:40.341476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bdc17ef1-3151-4734-b02d-0bda767cd318 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160395 Sep 19 16:44:40.342048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 984/1783] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:40 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:41.367027 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:41.367707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:41.374153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:41.374650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:41.375082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:41.375762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:41.376355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:41.377146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:41.514735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:41.523178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7773b316-0aaf-4625-b93d-b513df09d6fa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159906 Sep 19 16:44:41.523762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 985/1784] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:42.561430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:42.562098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:42.568325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:42.568798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:42.569233 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:42.569916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:42.570363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:42.570805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:42.700234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:42.709204 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d094528a-8ee0-4565-afe4-e6bcf0c9483b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151747 Sep 19 16:44:42.709782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 986/1785] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:43.733023 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:43.733661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:43.740560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:43.741035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:43.741467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:43.742151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:43.742632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:43.743054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:43.876928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:43.885069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e0237fc2-e856-4a9c-8d35-c9c1665132e6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155917 Sep 19 16:44:43.885634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 987/1786] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:43 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:44.909676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:44.910407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:44.916445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:44.916912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:44.917334 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:44.918008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:44.918482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:44.918898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:45.064141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:45.072357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c3ad207e-0ac9-4a3d-b291-b046f0190ad2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.167162 Sep 19 16:44:45.072926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 988/1787] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:44 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:46.096032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:46.096738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:46.103144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:46.103624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:46.104061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:46.104756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:46.105207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:46.105629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:46.239600 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:46.247815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-edcb28e6-9273-461d-9cdc-c787e227e875 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155782 Sep 19 16:44:46.248385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 989/1788] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:46 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:47.271387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:47.272082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:47.278483 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:47.278963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:47.279400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:47.280093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:47.280547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:47.280965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:47.409607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:47.417803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2a92cc94-618c-4b7f-b635-434814d6a5de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150381 Sep 19 16:44:47.418454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 990/1789] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:48.442411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:48.443063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:48.449288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:48.449762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:48.450194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:48.450905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:48.451354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:48.451771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:48.579676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:48.587850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8151a20f-cb55-4100-91a7-8a9539fb7be4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149967 Sep 19 16:44:48.588408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 991/1790] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:49.610824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:49.611512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:49.617895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:49.618305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:49.618764 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:49.619457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:49.619901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:49.620321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:49.751715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:49.759928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b49876ea-6978-4a24-a014-c736d2dde2c8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152923 Sep 19 16:44:49.760506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 992/1791] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:50.784812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:50.785702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:50.792533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:50.793013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:50.793449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:50.794139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:50.794637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:50.795064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:50.924855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:50.933345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9155f2e2-105c-4678-a5af-1722499dbc59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153193 Sep 19 16:44:50.933912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 993/1792] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:50 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:51.957001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:51.957659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:51.964428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:51.964909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:51.965348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:51.966051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:51.966535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:51.966965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:52.103360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:52.111752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4946bb58-9425-4a69-92e5-e4887684011e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159153 Sep 19 16:44:52.112331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 994/1793] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:51 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:53.135598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:53.136242 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:53.143220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:53.143700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:53.144132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:53.144818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:53.145267 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:53.145690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:53.271504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:53.279784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f2d6c1de-1fbd-421b-a32d-2b9f5098b8ea tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148145 Sep 19 16:44:53.280352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 995/1794] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:53 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:54.304540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:54.305466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:54.311742 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:54.312215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:54.312644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:54.313318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:54.313756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:54.314208 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:54.446889 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:54.455029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a3591698-f7c0-4d41-89b4-79dfc092124a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155548 Sep 19 16:44:54.455585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 996/1795] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:55.478749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:55.479445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:55.486235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:55.486740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:55.487160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:55.487832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:55.488270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:55.488684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:55.506699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:55.618911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:55.627670 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-03748057-fad8-4461-ba7f-afd12d4a56f8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152790 Sep 19 16:44:55.628238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 997/1796] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:56.651677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:56.652382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:56.658761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:56.659235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:56.659671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:56.660364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:56.660881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:56.661313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:56.788134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:56.796308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-733763bd-0dd1-4dc6-86d9-c8d16c5d6e86 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149255 Sep 19 16:44:56.796880 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 998/1797] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:44:57.819585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:57.820322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:57.826770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:57.827241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:57.827676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:57.828860 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:57.829304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:57.829718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:57.962576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:57.971705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-63f57e20-d6ea-4c89-9fd2-afc563e7955d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156054 Sep 19 16:44:57.972289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 999/1798] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:57 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:44:58.994965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:44:58.995603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:44:58.997892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=52 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:44:59.003451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:59.003926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:59.004355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:59.005051 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:44:59.005490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:44:59.005942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:44:59.139249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:44:59.148088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-56957188-71d6-4345-8535-d9cbf366364c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157119 Sep 19 16:44:59.148660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1000/1799] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:44:58 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:00.172373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:00.173004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:00.179489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:00.180228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:00.180658 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:00.181346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:00.181791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:00.182243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:00.319784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:00.329168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-05a0e20c-d273-4576-92a1-0177326c078c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160755 Sep 19 16:45:00.329749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1001/1800] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:00 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:01.352892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:01.354496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:01.360948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:01.361420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:01.361844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:01.362555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:01.363000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:01.363417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:01.492509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:01.500753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6828d76f-c5e4-43b5-9043-b32f6f99035b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151963 Sep 19 16:45:01.501324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1002/1801] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:02.525628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:02.526312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:02.533689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:02.534228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:02.534699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:02.535395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:02.535848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:02.536281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:02.664503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:02.673602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b72e555c-4963-4b53-9c22-f4f012a9b115 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152466 Sep 19 16:45:02.674163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1003/1802] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:02 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:03.696357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:03.696976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:03.703399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:03.703869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:03.704322 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:03.705029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:03.705471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:03.705922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:03.838415 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:03.846605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f0a47d4b-e1e4-4aa3-9b4e-0fb40a57317b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154067 Sep 19 16:45:03.847169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1004/1803] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:04.870837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:04.871595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:04.878039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:04.878549 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:04.878982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:04.879674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:04.880117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:04.880530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:05.013388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:05.021705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4098b119-1ea0-41ff-808a-73e63126e4b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154704 Sep 19 16:45:05.022292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1005/1804] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:04 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:06.045463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:06.046174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:06.053063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:06.053543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:06.053968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:06.054692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:06.055392 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:06.055807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:06.199522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:06.207797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fbc682e9-8e99-4bb2-9d2c-916c0e78bdd8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.166538 Sep 19 16:45:06.208364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1006/1805] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:06 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:07.231271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:07.231987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:07.238479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:07.238961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:07.239400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:07.240129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:07.240629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:07.241102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:07.370849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:07.379638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-26c27be7-c20a-49b9-adda-80b7a2b270bd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152242 Sep 19 16:45:07.380225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1007/1806] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:07 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:08.402230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:08.402901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:08.409910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:08.410396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:08.410843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:08.412547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:08.413011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:08.413439 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:08.545117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:08.553532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e37448d8-5cb3-4810-b5ad-975da5080a8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155159 Sep 19 16:45:08.554124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1008/1807] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:08 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:09.577145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:09.577834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:09.584248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:09.584754 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:09.585190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:09.585875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:09.586324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:09.586772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:09.718115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:09.727389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-823c1d72-fbaa-474f-95b6-26c459c59993 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154047 Sep 19 16:45:09.727964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1009/1808] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:09 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:10.753285 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:10.753988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:10.760830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:10.761316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:10.761752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:10.762488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:10.762944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:10.763368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:10.901458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:10.910460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4572e5b8-645f-4b95-bbd1-2bea97ad01c3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160976 Sep 19 16:45:10.911055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1010/1809] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:10 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:11.936479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:11.937144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:11.944077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:11.944547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:11.944973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:11.945654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:11.946098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:11.946537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:12.082345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:12.090660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1a458b86-c0e3-44b7-92b7-d4606f1dcfbf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158048 Sep 19 16:45:12.091241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1011/1810] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:11 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:13.113617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:13.114286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:13.120887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:13.121364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:13.121800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:13.122527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:13.122983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:13.123404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:13.142582 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'services' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:45:13.260299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:13.277539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6f2e7977-c4f4-4703-ac7c-db86846be306 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.167895 Sep 19 16:45:13.278153 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1012/1811] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:13 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:14.301309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:14.302028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:14.309174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:14.309653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:14.310087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:14.310812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:14.311264 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:14.311691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:14.448476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:14.457299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-02256085-cd36-49be-96bd-27a52eef9ab4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159917 Sep 19 16:45:14.457879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1013/1812] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:14 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:15.480741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:15.481394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:15.488673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:15.489148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:15.489588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:15.490278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:15.490766 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:15.491191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:15.495132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=100 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:15.622028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:15.630326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1de83db7-4a9e-423e-8e2b-2eca57abce3b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153479 Sep 19 16:45:15.630929 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1014/1813] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:15 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:16.653995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:16.655540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:16.662244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:16.662756 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:16.663194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:16.663887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:16.664337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:16.664757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:16.795954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:16.805163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-888eb8a9-2714-482f-9027-e8fb3fb3fcad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154729 Sep 19 16:45:16.805743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1015/1814] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:16 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:17.828367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:17.829599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:17.835976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:17.836447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:17.836879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:17.837565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:17.838012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:17.838466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:17.967346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:17.975488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-174b8dfd-19c0-4b58-9f18-4c081a7db31f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151090 Sep 19 16:45:17.976050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1016/1815] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:18.999372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:19.000065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:19.006962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:19.007434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:19.007869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:19.008551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:19.008992 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:19.009405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:19.145035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:19.153736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f5f80a6a-0ed2-422b-9e50-0b5294ac8936 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158117 Sep 19 16:45:19.154336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1017/1816] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:18 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:19.564618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 HTTP/1.1 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:45:19.566866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:45:19.568702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'server_id': '4afae149-ed22-4a06-a4b0-d87d726d834f', 'id': '741cfdcd-bd82-4d3e-8f21-5f862f2381f9'} Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:45:19.569807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.BadRequest: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5194, in _detach_volume Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.begin_detaching(context, volume['id']) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 403, in wrapper Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.InvalidInput(reason=err_msg)) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:45:19.571230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 560, in begin_detaching Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.begin_detaching(volume_id) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 310, in begin_detaching Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._action('os-begin_detaching', volume) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 257, in _action Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.post(url, body=body) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 227, in post Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'POST', **kwargs) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidInput: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:45:19.573108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5216, in detach_volume Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._detach_volume(context, instance, volume) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 231, in wrapped Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5196, in _detach_volume Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=exc.format_message()) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:45:19.574452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 567, in delete Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) Sep 19 16:45:19.576088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 400 to user: Invalid volume: Invalid input received: Invalid volume: Unable to detach volume. Volume status must be 'in-use' and attach_status must be 'attached' to detach. (HTTP 400) (Request-ID: req-f7a8bdbe-18b7-4b73-a37e-b4250d3e9439) {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9" status: 400 len: 269 microversion: 2.1 time: 119.961867 Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:45:19 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (ip 10.222.0.57) !!! Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:45:19 2022 - uwsgi_response_writev_headers_and_body_do(): Broken pipe [core/writer.c line 306] during DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 (10.222.0.57) Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: CRITICAL nova [None req-7915d90b-cb16-4a8f-8509-76a6c0e32837 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Unhandled error: OSError: write error Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova OSError: write error Sep 19 16:45:19.577488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova Sep 19 16:45:19.578183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 800/1817] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:43:19 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f/os-volume_attachments/741cfdcd-bd82-4d3e-8f21-5f862f2381f9 => generated 0 bytes in 119963 msecs (HTTP/1.1 400) 9 headers in 0 bytes (0 switches on core 0) Sep 19 16:45:20.179154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:20.179769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:20.186608 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:20.187088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:20.187521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:20.188866 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:20.189316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:20.189741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:20.327618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:20.335943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a3c02c59-a0f6-4b8e-b332-6d01c0dc0d07 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160997 Sep 19 16:45:20.336519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1018/1818] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:21.361068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:21.361677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:21.367716 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:21.368186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:21.368617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:21.369301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:21.369751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:21.370168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:21.508339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:21.517905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e5e2c8fd-5ba4-4310-9a8a-ee95d5c3f83b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.162514 Sep 19 16:45:21.518497 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 801/1819] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:21 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:22.541230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:22.541834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:22.548832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:22.549321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:22.549759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:22.550623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:22.551084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:22.551512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:22.687098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:22.695574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2c055934-df56-401f-9470-b3e9d5021f05 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158259 Sep 19 16:45:22.696167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1019/1820] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:23.720500 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:23.721160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:23.727978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:23.728804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:23.728951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:23.729606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:23.730084 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:23.730522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:23.878940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:23.887400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0c0f3a2f-1531-48fe-b2e2-60f0c5d907c7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.171532 Sep 19 16:45:23.887894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 802/1821] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:23 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:24.911173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:24.911780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:24.918447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:24.919049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:24.919476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:24.920169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:24.920619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:24.921036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:25.060399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:25.068963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-79a844f5-9882-442d-b72f-cdc24e132f1d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161598 Sep 19 16:45:25.069545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1020/1822] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:26.093581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:26.094224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:26.100833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:26.101318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:26.101754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:26.102506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:26.102963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:26.103391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:26.272520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:26.281696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a95e16fb-3fd2-4e63-bd06-1eee6b85599a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.192024 Sep 19 16:45:26.282272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 803/1823] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:27.305711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:27.306333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:27.313890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:27.314399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:27.314840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:27.315557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:27.316008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:27.316434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:27.452741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:27.461851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-380e53ea-7fc3-4d5f-aac0-188adeb13e02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160407 Sep 19 16:45:27.462801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1021/1824] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:27 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:28.485923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:28.486578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:28.493638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:28.494121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:28.494600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:28.495319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:28.496080 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:28.496509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:28.641385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:28.650012 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-df239225-a240-4a01-9598-c829f3effeb9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.168047 Sep 19 16:45:28.650654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 804/1825] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:29.675340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:29.675961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:29.677919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=44 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:29.683731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:29.684212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:29.684646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:29.685351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:29.685800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:29.686227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:29.821737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:29.830272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-556c3c7c-eba2-49bf-a2a6-5eb6304d7b77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159614 Sep 19 16:45:29.830892 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1022/1826] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:30.853358 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:30.854089 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:30.861263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:30.861749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:30.862185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:30.862918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:30.863375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:30.863802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:31.015914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:31.025160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-71d92871-125e-4477-b396-2e44b733e551 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.175696 Sep 19 16:45:31.025768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 805/1827] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:30 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:32.049315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:32.050018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:32.056655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:32.057135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:32.057574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:32.058270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:32.058779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:32.059202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:32.194759 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:32.203432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9ef9b801-6b94-4abc-8d59-19269a94d1b9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158103 Sep 19 16:45:32.204012 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1023/1828] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:32 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:33.227163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:33.227784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:33.234794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:33.235282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:33.235721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:33.236433 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:33.236889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:33.237318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:33.374339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:33.383428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e32d35a6-c16a-40d5-b0f1-4c8851dc595e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160113 Sep 19 16:45:33.384015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 806/1829] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:34.407693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:34.408294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:34.410335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:45:34.410565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:45:34 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:45:34.422268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:34.422788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:34.423218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:34.423914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:34.424354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:34.424773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:34.568490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:34.577841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f2f2146f-4337-4349-a4e2-f80a47ce1c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.174648 Sep 19 16:45:34.578980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1024/1830] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:34 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:35.603221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:35.604009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:35.611338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:35.611826 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:35.612260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:35.612969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:35.613421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:35.613842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:35.751274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:35.759829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-850a3e8a-e585-4b0c-b5b2-ec364bb008dd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160497 Sep 19 16:45:35.760640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 807/1831] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:36.787038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:36.788157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:36.800432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:36.801272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:36.802029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:36.803348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:36.804091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:36.804674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:36.960686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:36.969945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a199031f-e4f4-4b24-b4b2-43f91e5dc854 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.190082 Sep 19 16:45:36.970565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1025/1832] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:36 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:37.995584 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:37.996205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:38.004152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:38.004773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:38.005254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:38.006066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:38.006631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:38.007106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:38.157248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:38.166116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d65a3c21-835f-4e6e-be2d-62347e2fe588 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.174382 Sep 19 16:45:38.166743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 808/1833] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:45:39.190545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:39.191152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:39.197551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:39.198034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:39.198509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:39.199241 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:39.199752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:39.200184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:39.337180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:39.345547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d4463062-9dfe-412f-9e90-a178f33ee00f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159035 Sep 19 16:45:39.346117 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1026/1834] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:39 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:40.368682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:40.369364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:40.376042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:40.376555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:40.376961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:40.377770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:40.378258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:40.378715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:40.381400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=53 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:40.511972 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:40.520760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b2dd9eb4-c96b-4781-b760-21e3fa6f454e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156174 Sep 19 16:45:40.521327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 809/1835] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:40 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:41.545395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:41.546481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:41.555753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:41.556234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:41.556678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:41.557378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:41.557829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:41.558253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:41.697481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:41.705698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-102cd8c9-e8e1-42ba-b433-9e094b069ffd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.165118 Sep 19 16:45:41.706259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1027/1836] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:42.729125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:42.729709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:42.736603 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:42.737068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:42.737491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:42.738167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:42.738641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:42.739059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:42.869947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:42.878654 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-89f92b17-3a55-41a2-82de-100102aa114d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153339 Sep 19 16:45:42.879214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 810/1837] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:43.902672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:43.903231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:43.909650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:43.910107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:43.910564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:43.911258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:43.911708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:43.912124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:44.047599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:44.056422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a988ce60-be54-48c7-a873-c5dbdfd47c52 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157741 Sep 19 16:45:44.057014 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1028/1838] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:43 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:45.080929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:45.081473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:45.087932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:45.088418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:45.088855 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:45.089547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:45.090000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:45.090464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:45.219253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:45.227484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2d8b7df9-8ffa-4fa5-8853-a8992ebd0c7a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151009 Sep 19 16:45:45.228051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 811/1839] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:45 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:46.250399 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:46.251018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:46.257453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:46.257934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:46.258386 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:46.259086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:46.259544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:46.259971 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:46.262625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=89 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:46.388576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:46.397143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e9e0fcb6-e8c6-49fd-a0f4-2764a7acbfc9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150876 Sep 19 16:45:46.397711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1029/1840] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:46 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:47.420651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:47.421266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:47.427994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:47.428476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:47.428910 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:47.429605 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:47.430310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:47.430773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:47.569920 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:47.578480 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aadf8bc0-0642-4970-a9ce-bb32c029c697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161571 Sep 19 16:45:47.578982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 812/1841] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:48.602777 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:48.603429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:48.610531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:48.611079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:48.611517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:48.612210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:48.612659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:48.613073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:48.748667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:48.757552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f111c9b6-7f4d-4662-ba91-32af4e89c5a9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158638 Sep 19 16:45:48.758175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1030/1842] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:49.781562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:49.781971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:49.788840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:49.789316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:49.789745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:49.790474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:49.790924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:49.791341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:49.928009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:49.936744 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-906224c3-2a48-4966-889f-f9c892e72c40 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159682 Sep 19 16:45:49.937859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 813/1843] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:50.960408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:50.961050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:50.967516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:50.967996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:50.969111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:50.969807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:50.970261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:50.970731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:51.098580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:51.106833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aed983a4-902c-46f3-9f0e-391a73dcfe6b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150351 Sep 19 16:45:51.107394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1031/1844] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:50 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:52.134924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:52.135555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:52.137933 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=27 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:45:52.143289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:52.143767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:52.144193 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:52.144884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:52.145330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:52.145747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:52.277491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:52.286303 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cd382ba1-b54a-4665-af90-43958d84b87a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155204 Sep 19 16:45:52.286928 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 814/1845] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:52 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:53.310433 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:53.311025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:53.318411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:53.318829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:53.319258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:53.319937 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:53.320374 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:53.320791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:53.449055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:53.457206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-33285eb6-0593-4581-b853-df3de09f57c4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150677 Sep 19 16:45:53.457799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1032/1846] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:53 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:54.481326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:54.481954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:54.488496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:54.488983 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:54.489423 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:54.490119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:54.490597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:54.491047 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:54.626486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:54.638082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2cbbaca2-d0f1-4cd5-833a-f5fbf73b53af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160588 Sep 19 16:45:54.638706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 815/1847] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:55.661686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:55.662300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:55.668752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:55.669228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:55.669660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:55.670343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:55.670823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:55.671245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:55.801652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:55.810130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b509134f-b05c-4e7f-856f-5bc46800af35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152349 Sep 19 16:45:55.810720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1033/1848] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:56.833976 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:56.834635 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:56.837021 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:45:56.837151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:45:56 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:45:56.842823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:56.843786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:56.844215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:56.844905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:56.845354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:56.845777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:56.973234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:56.981332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-320ccfef-336a-4649-866e-e097754c6571 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151264 Sep 19 16:45:56.981892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 816/1849] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:58.003698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:58.004292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:58.011197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:58.011675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:58.012112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:58.012804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:58.013252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:58.013682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:58.148381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:58.157337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b50bfbab-749e-438f-8150-864caae261ce tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157530 Sep 19 16:45:58.157906 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1034/1850] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:57 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:45:59.181371 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:45:59.181903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:45:59.188913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:59.189381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:59.189808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:59.190521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:45:59.190966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:45:59.191384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:45:59.324018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:45:59.332323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fcb3bcb8-e7f3-463a-ab89-fd87173f13ad tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155116 Sep 19 16:45:59.332894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 817/1851] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:45:59 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:00.356196 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:00.356946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:00.358896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:00.365272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:00.365748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:00.366180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:00.366901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:00.367344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:00.367761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:00.497859 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:00.506557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ed74bb10-d443-442f-97f2-05f70a7ecf57 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154546 Sep 19 16:46:00.507135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1035/1852] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:00 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:01.530385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:01.531130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:01.538042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:01.538558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:01.539001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:01.539699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:01.540156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:01.540581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:01.667459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:01.675720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-79307913-5c94-423d-bee8-f7ed1ad68468 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149108 Sep 19 16:46:01.676143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 818/1853] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:02.698973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:02.699584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:02.705964 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:02.706473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:02.706912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:02.707602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:02.708049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:02.708471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:02.836800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:02.845508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4d4dfcfd-397f-49bc-9460-72f4b741b14c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150468 Sep 19 16:46:02.846072 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1036/1854] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:02 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:03.870310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:03.870955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:03.878136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:03.878652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:03.879091 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:03.879785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:03.880240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:03.880667 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:04.009866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:04.018264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b9993010-6ce6-40c1-abf0-d1f64130a852 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151854 Sep 19 16:46:04.018866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 819/1855] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:05.042300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:05.042850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:05.049441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:05.049915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:05.050343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:05.051076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:05.051533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:05.051957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:05.187666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:05.195993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f5257929-c989-4e0f-9cbf-f2d2a2212c00 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157655 Sep 19 16:46:05.196564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1037/1856] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:05 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:06.220852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:06.221473 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:06.227837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:06.228315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:06.228777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:06.229465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:06.229914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:06.230339 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:06.368044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:06.376754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5f9484c8-beb1-41d9-b1df-6b2d9f085ab3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160759 Sep 19 16:46:06.377328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 820/1857] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:06 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:07.400539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:07.401149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:07.408268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:07.408750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:07.409187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:07.409878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:07.410326 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:07.410780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:07.539190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:07.547407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f156d8b7-8e92-4a90-b85a-cce3eecf74f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150815 Sep 19 16:46:07.547975 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1038/1858] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:07 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:08.570832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:08.571420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:08.577871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:08.578343 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:08.578798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:08.579474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:08.579936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:08.580334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:08.708351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:08.716613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c55922fe-730e-4549-b49e-d7b643207f19 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149568 Sep 19 16:46:08.717177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 821/1859] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:08 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:09.740500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:09.741147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:09.747659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:09.748138 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:09.748575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:09.749835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:09.750288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:09.750737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:09.877747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:09.885825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c2b27b71-b4e8-4d07-bb63-5b87769c118d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149282 Sep 19 16:46:09.886397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1039/1860] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:09 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:10.910083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:10.910741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:10.917306 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:10.917721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:10.918151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:10.918884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:10.919329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:10.919749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:10.923664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:11.048068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:11.056524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-41a37319-9d36-498c-9789-f884d95990fb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151209 Sep 19 16:46:11.057094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 822/1861] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:10 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:12.080966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:12.081556 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:12.088068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:12.088537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:12.088963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:12.089637 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:12.090080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:12.090527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:12.225421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:12.234352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d31b399b-5a0c-4569-9669-13ba760f290e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157662 Sep 19 16:46:12.234960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1040/1862] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:12 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:13.442145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:13.442795 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:13.448731 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:13.449207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:13.449639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:13.450327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:13.450811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:13.451229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:13.645942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:13.654270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-46fc3185-7f3b-4994-9ccd-fcc8113911bf tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.399728 Sep 19 16:46:13.654886 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 823/1863] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:13 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 401 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:14.678030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:14.678610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:14.685195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:14.685662 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:14.686088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:14.686803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:14.687247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:14.687656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:14.815265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:14.823441 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7a6dc971-3903-472b-a4eb-cc6304b9cfac tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149306 Sep 19 16:46:14.824217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1041/1864] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:14 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:15.847465 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:15.848086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:15.854659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:15.855131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:15.855568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:15.856289 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:15.856739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:15.857161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:15.986098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:15.994249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4900c2fe-3900-47e3-b70e-f7c1da6fd6e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150692 Sep 19 16:46:15.994845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 824/1865] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:15 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:17.017925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:17.018566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:17.025047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:17.025526 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:17.025959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:17.026680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:17.027135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:17.027561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:17.030141 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:17.158808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:17.166931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c57bc0e7-4810-499d-b894-713a661b71d8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153530 Sep 19 16:46:17.167505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1042/1866] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:18.190217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:18.190871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:18.196988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:18.197469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:18.197988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:18.198629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:18.199087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:18.199515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:18.326165 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:18.334890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-02c1d275-6449-4b43-a310-9c5d4aa7f550 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148772 Sep 19 16:46:18.335452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 825/1867] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:18 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:19.357780 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:19.358438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:19.364819 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:19.365289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:19.365720 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:19.366454 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:19.366907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:19.367329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:19.494535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:19.502630 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8d68f497-922d-4ec4-98cb-632b5140ac66 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148684 Sep 19 16:46:19.503198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1043/1868] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:19 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:20.526086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:20.526710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:20.533714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:20.534191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:20.534693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:20.535385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:20.535845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:20.536273 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:20.678206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:20.686816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b3de2f12-5980-4808-b345-b74a8e15c68e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.164714 Sep 19 16:46:20.687401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 826/1869] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:21.709139 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:21.709743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:21.716265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:21.716740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:21.717167 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:21.717843 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:21.718284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:21.718741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:21.848816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:21.857092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7d5f0626-5d8d-4a8c-97e7-dda9512c2805 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151854 Sep 19 16:46:21.857652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1044/1870] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:21 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:22.880946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:22.881562 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:22.883956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:22.890037 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:22.890551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:22.890992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:22.891686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:22.892142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:22.892568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:23.020382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:23.028604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1d45368b-9990-4f12-a992-79a4ac912d77 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151645 Sep 19 16:46:23.029234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 827/1871] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:24.051202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:24.051748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:24.058527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:24.058999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:24.059427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:24.060100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:24.060541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:24.060959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:24.192438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:24.200645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e82d71c8-95c1-4a92-a923-d4bbc29b84e4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153358 Sep 19 16:46:24.201215 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1045/1872] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:25.224187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:25.224825 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:25.231174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:25.231646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:25.232082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:25.232765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:25.233233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:25.233651 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:25.362756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:25.370807 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d9b2c51f-43d7-4995-8168-23bcda927245 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150502 Sep 19 16:46:25.371382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 828/1873] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:25 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:26.396265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:26.396851 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:26.403236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:26.403709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:26.404135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:26.404858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:26.405790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:26.406205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:26.534562 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:26.542623 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-28d4e021-4042-4c6f-a0cc-ef1ce44995a5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150221 Sep 19 16:46:26.543188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1046/1874] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:27.566726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:27.567355 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:27.579036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:27.579520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:27.579963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:27.580669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:27.581123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:27.581553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:27.709469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:27.717745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1f89d859-8ef5-4332-bc60-7f671b8e7fd7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155010 Sep 19 16:46:27.718314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 829/1875] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:27 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:28.741400 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:28.741994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:28.749175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:28.749653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:28.750085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:28.750803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:28.751252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:28.751678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:28.888106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:28.896816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-048292e8-816a-4294-81f3-71b0bc4a042e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159306 Sep 19 16:46:28.897387 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1047/1876] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:29.919567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:29.920214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:29.926556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:29.926967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:29.927395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:29.928072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:29.928512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:29.928926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:30.054156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:30.062659 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9b7c1c26-8529-467b-8bbf-07ed21cd554c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.146977 Sep 19 16:46:30.063228 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 830/1877] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:31.086788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:31.087335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:31.089337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:31.094856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:31.095340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:31.095776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:31.096470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:31.096924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:31.097379 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:31.223810 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:31.232043 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0f8c64dd-aa3a-4396-9ccd-1969e14a3d0f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149264 Sep 19 16:46:31.232612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1048/1878] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:31 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:46:32.256093 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:32.256713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:32.263232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:32.263712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:32.264146 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:32.264841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:32.265291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:32.266207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:32.401675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:32.414405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dabe6664-c5cc-4014-8202-a3f100baf396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.162751 Sep 19 16:46:32.414984 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 831/1879] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:32 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:33.437430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:33.438025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:33.444393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:33.444855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:33.445279 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:33.445951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:33.446414 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:33.446834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:33.576822 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:33.585395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1b716f2a-4869-4a2b-b4af-a7da31561373 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151853 Sep 19 16:46:33.585951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1049/1880] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:34.609895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:34.610534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:34.616945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:34.617420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:34.617854 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:34.618566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:34.619019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:34.619445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:34.745876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:34.754044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dc0c7fd3-9dd2-4977-b153-daf04c46fed4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148639 Sep 19 16:46:34.754639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 832/1881] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:34 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:35.777158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:35.777768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:35.784470 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:35.784945 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:35.785372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:35.786048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:35.786528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:35.786942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:35.913501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:35.922108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fed2fd1f-0f87-49ab-8b71-183a1fb7fe88 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148352 Sep 19 16:46:35.922732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1050/1882] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:37.009191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:37.009794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:37.016293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:37.017413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:37.017847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:37.018587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:37.020832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:37.021348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:37.153120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:37.161319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f2771a7d-d671-41c5-bb55-e008570a17b0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156079 Sep 19 16:46:37.161888 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 833/1883] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:38.185078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:38.185676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:38.192324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:38.192837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:38.193269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:38.193963 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:38.194442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:38.194868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:38.327362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:38.336110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38b1af80-0cbf-455a-b7e0-f515793c618d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154899 Sep 19 16:46:38.336714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1051/1884] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:38 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:39.360868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:39.361519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:39.368205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:39.368682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:39.369116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:39.369802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:39.370255 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:39.370710 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:39.497941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:39.506284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b2d67dd0-205c-48a5-85f4-18ce56eac44d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149883 Sep 19 16:46:39.506894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 834/1885] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:39 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:40.530197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:40.530815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:40.537253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:40.537728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:40.538166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:40.538873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:40.539320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:40.539743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:40.670545 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:40.679013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e9f1cd0a-fe9d-4371-949b-b3ac26075e90 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152668 Sep 19 16:46:40.679582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1052/1886] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:40 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:41.703506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:41.704284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:41.711504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:41.714333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:41.714333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:41.714333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:41.714333 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:41.714834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:41.717263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:41.858845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:41.870349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c94d408b-85ef-47fb-a848-558ceb56ba6a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.171436 Sep 19 16:46:41.871011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 835/1887] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:42.894617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:42.895234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:42.901655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:42.902144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:42.902618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:42.903309 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:42.903760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:42.904186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:43.034571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:43.042821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-641e977b-013d-4366-851e-4c457774a8ab tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152064 Sep 19 16:46:43.043391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1053/1888] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:44.067483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:44.068102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:44.074567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:44.075050 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:44.075486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:44.076181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:44.076634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:44.077100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:44.209799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:44.217833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9981b4a9-9e6d-4f91-8aae-3d5de5f396c5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154234 Sep 19 16:46:44.218410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 836/1889] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:44 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:46:45.241858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:45.242507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:45.248913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:45.249394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:45.249833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:45.250544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:45.251488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:45.251911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:45.382538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:45.390747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e7dbbe7b-67a1-4093-bd6f-af95af6890de tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152820 Sep 19 16:46:45.391311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1054/1890] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:45 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:46.414752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:46.415389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:46.421754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:46.422234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:46.422741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:46.423430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:46.423889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:46.424315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:46.552780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:46.561064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-254f60a2-bef3-4843-92b9-cf85bddbce75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150273 Sep 19 16:46:46.561616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 837/1891] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:46 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:47.583634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:47.584234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:47.590794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:47.591270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:47.591701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:47.592383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:47.592831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:47.593253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:47.596304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:47.724428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:47.734546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0facdbf4-a1a4-4d2a-b511-0a65272d2f75 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154570 Sep 19 16:46:47.735757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1055/1892] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:48.758315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:48.758980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:48.765226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:48.765706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:48.766141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:48.766877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:48.767334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:48.767754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:48.898266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:48.906704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1ff81d05-7f28-46dd-8305-5c70e4ca8697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152259 Sep 19 16:46:48.907755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 838/1893] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:49.931271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:49.931960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:49.938530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:49.938980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:49.939417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:49.940108 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:49.940572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:49.940986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:50.070757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:50.079667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4c00920a-b119-49fe-9118-25030d226bf5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152039 Sep 19 16:46:50.080250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1056/1894] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:51.103648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:51.104204 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:51.111260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:51.111740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:51.112176 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:51.112869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:51.113337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:51.113747 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:51.242956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:51.251129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4a286a63-12c5-438b-a750-4a0111f08581 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151482 Sep 19 16:46:51.251694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 839/1895] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:51 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:52.274853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:52.275491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:52.281856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:52.282332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:52.282870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:52.283557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:52.284005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:52.284481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:52.409834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:52.418028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2351c5b5-801e-448c-8447-65e128a06b2c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.147228 Sep 19 16:46:52.418634 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1057/1896] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:52 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:53.442748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:53.443623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:53.445858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:46:53.451470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:53.451957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:53.452391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:53.453095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:53.453544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:53.454565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:53.585843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:53.593964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-087521a4-1297-492c-99b5-9c8a7ee4f8f4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155610 Sep 19 16:46:53.594557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 840/1897] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:53 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:54.617163 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:54.617729 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:54.624303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:54.624781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:54.625217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:54.625911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:54.626360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:54.626808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:54.757018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:54.765587 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b74c385f-04c3-4bda-98a1-9e4b035793d1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152268 Sep 19 16:46:54.766087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1058/1898] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:55.789066 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:55.789674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:55.796109 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:55.796576 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:55.797009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:55.798334 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:55.798818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:55.799233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:55.924760 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:55.933166 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0a305eb9-22d2-435f-8df3-c887d69b0040 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.147994 Sep 19 16:46:55.933729 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 841/1899] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:56.955444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:56.956102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:56.962507 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:56.962980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:56.963416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:56.964093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:56.964538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:56.964958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:57.347169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:57.355265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-befb8cd0-dbd5-45d0-b402-709237483916 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.403749 Sep 19 16:46:57.355826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1059/1900] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 405 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:58.379764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:58.380314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:58.386706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:58.387185 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:58.388116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:58.389217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:58.389675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:58.390103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:58.516988 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:58.525132 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7b5b5508-a0ec-40f1-8c1e-0b40c83eba0b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149913 Sep 19 16:46:58.525706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 842/1901] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:58 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:46:59.548794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:46:59.549406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:46:59.555721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:59.556199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:59.556627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:59.557303 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:46:59.557748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:46:59.558172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:46:59.685262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:46:59.693882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0115472c-739f-4091-9f53-10466970fc6d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149164 Sep 19 16:46:59.694467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1060/1902] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:46:59 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:00.718008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:00.718671 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:00.725697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:00.726173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:00.726638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:00.727319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:00.727767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:00.728187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:00.857385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:00.865556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-de8e17bc-70de-468a-a971-5df15f4413d3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151368 Sep 19 16:47:00.866125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 843/1903] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:00 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:01.889046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:01.889715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:01.892038 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:01.897174 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:01.897671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:01.898099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:01.898805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:01.899250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:01.899661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:02.034844 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:02.043019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-600f24e9-3891-4d51-aeb1-275f1adeebc4 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157954 Sep 19 16:47:02.043590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1061/1904] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:03.066573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:03.067133 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:03.073560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:03.074042 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:03.074513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:03.075217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:03.075674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:03.076100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:03.204580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:03.212763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9f1eaec4-5b21-4d51-be5f-6536dff5e98d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150184 Sep 19 16:47:03.213351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 844/1905] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:04.239864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:04.240453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:04.247206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:04.247680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:04.248137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:04.248803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:04.249253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:04.249672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:04.383160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:04.391899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8ec0fe87-6c56-41e7-9405-e6a0a985befd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155426 Sep 19 16:47:04.392466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1062/1906] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:04 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:05.416011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:05.416611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:05.423620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:05.424116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:05.424577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:05.425272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:05.425725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:05.426148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:05.558291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:05.566831 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-098d1028-45e0-4ad7-bf8d-f953ed833add tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154481 Sep 19 16:47:05.567404 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 845/1907] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:05 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:06.590323 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:06.591064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:06.600166 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:06.600701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:06.601300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:06.602765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:06.603229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:06.603661 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:06.747552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:06.756563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a5252627-e1ed-4032-96c7-4e307c6bae42 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.170580 Sep 19 16:47:06.757165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1063/1908] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:06 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:07.780412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:07.781058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:07.789329 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:07.789813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:07.790251 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:07.790990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:07.791445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:07.791876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:07.997501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:08.006538 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aa749758-adde-48af-bf34-ff427c226db6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.229854 Sep 19 16:47:08.007117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 846/1909] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:07 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:09.031302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:09.032489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:09.038956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:09.039432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:09.039872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:09.040559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:09.041011 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:09.041435 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:09.169739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:09.177841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8bb2cf46-4898-4d3e-8234-4c4b6eb99f25 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150674 Sep 19 16:47:09.178422 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1064/1910] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:09 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:10.201923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:10.202589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:10.204712 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:47:10.205363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:47:10 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:47:10.213559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:10.214035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:10.214509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:10.215246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:10.215680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:10.216095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:10.347948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:10.356223 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7c9948fc-c5ec-4e2e-b196-5e3ac6b1f005 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158210 Sep 19 16:47:10.356793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 847/1911] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:10 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:11.380282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:11.380897 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:11.388456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:11.388456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:11.388456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:11.389090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:11.389542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:11.389967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:11.516176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:11.524372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f0eaee1a-c37b-4c60-9bc0-c12476096718 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148000 Sep 19 16:47:11.524940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1065/1912] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:11 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:12.547195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:12.547751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:12.554098 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:12.554625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:12.555059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:12.556008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:12.556463 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:12.556893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:12.559445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:12.689604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:12.697788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c9fc1f29-8ef6-4b2b-bcb6-6bcc13c3960a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154535 Sep 19 16:47:12.698362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 848/1913] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:12 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:13.721709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:13.722319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:13.728925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:13.729401 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:13.729833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:13.730553 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:13.730998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:13.731412 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:13.891681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:13.903516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5bf4d71e-d7a2-426d-a6e3-d55cc28cd788 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.185333 Sep 19 16:47:13.904116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1066/1914] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:13 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:14.926309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:47:14.926869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:47:14 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:47:14.938691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:14.939375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:14.945277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:14.945751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:14.946178 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:14.946911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:14.947381 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:14.947799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:15.078422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:15.086677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-737930ad-ae48-4a03-bbdd-a23600004065 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161363 Sep 19 16:47:15.087241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 849/1915] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:14 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:16.112083 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:16.113345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:16.122245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:16.123201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:16.123934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:16.125524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:16.126251 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:16.126913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:16.271512 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:16.279748 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c410d421-5255-4e43-a3e5-d56fc79a9153 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.172805 Sep 19 16:47:16.280317 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1067/1916] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:16 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:17.304253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:17.304880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:17.313453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:17.313929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:17.315074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:17.316147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:17.316775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:17.317408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:17.455432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:17.463885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a73067d0-3174-40bb-a6c7-54405e0f9aed tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.163603 Sep 19 16:47:17.464449 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 850/1917] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:18.488797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:18.489510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:18.496501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:18.497068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:18.497568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:18.498336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:18.498871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:18.499366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:18.504030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:18.671149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:18.684716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-52254f38-4f1e-4a87-a5e2-db1be173151c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.200201 Sep 19 16:47:18.685911 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1068/1918] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:18 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:47:19.709202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:19.709845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:19.717060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:19.717563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:19.718009 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:19.721919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:19.721919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:19.721919 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:19.866915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:19.873168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f35f4101-fc63-4e2f-a3bc-55dc2be94881 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.168278 Sep 19 16:47:19.873757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 851/1919] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:19 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:20.897403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:20.898107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:20.904761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:20.905270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:20.905683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:20.907047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:20.907721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:20.908154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:21.048477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:21.057789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6fde4c52-00d8-4e9d-8ef2-48c0f7ba885a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.164164 Sep 19 16:47:21.058398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1069/1920] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:22.083436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:22.084051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:22.090528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:22.090948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:22.091395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:22.092081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:22.092536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:22.092963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:22.236874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:22.251660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bbb10305-4b5a-4893-a371-dc0b79f20ea6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.171699 Sep 19 16:47:22.252622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 852/1921] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:23.276350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:23.276965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:23.283446 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:23.283920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:23.284351 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:23.285032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:23.285481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:23.285907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:23.412150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:23.420429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-295cf094-910c-4a83-9e6a-d80eba2dcd59 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.148038 Sep 19 16:47:23.420999 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1070/1922] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:23 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:24.444495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:24.445112 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:24.447485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:24.452783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:24.453808 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:24.454243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:24.454961 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:24.455416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:24.455857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:24.584357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:24.592556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-87da924e-c6ca-4ab4-8705-57989b00d624 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151895 Sep 19 16:47:24.593116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 853/1923] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:25.616214 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:25.616825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:25.623275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:25.623785 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:25.624219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:25.625429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:25.625879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:25.626306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:25.760972 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:25.770165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bf5f2bdb-7b2b-4167-a92f-61fb3b4c07f3 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157823 Sep 19 16:47:25.770771 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1071/1924] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:25 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:26.793007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:26.793625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:26.800683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:26.801159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:26.801597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:26.802288 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:26.802771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:26.803220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:26.934683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:26.943432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-aa614cfc-db8f-48d9-a57f-a62cf5372d62 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154281 Sep 19 16:47:26.943998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 854/1925] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:27.966774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:27.967390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:27.973775 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:27.974259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:27.974741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:27.975438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:27.975894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:27.976324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:28.114448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:28.123030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-66f75abe-7708-4b5b-b655-dda07c8917a1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159984 Sep 19 16:47:28.123626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1072/1926] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:27 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:29.147431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:29.148060 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:29.154706 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:29.155268 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:29.155696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:29.156916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:29.157440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:29.157897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:29.285591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:29.293749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d875255c-abf0-4e1a-a925-4a58f0fa8c17 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150256 Sep 19 16:47:29.294315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 855/1927] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:30.316331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:30.317144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:30.323245 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:30.323727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:30.324160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:30.324852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:30.325300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:30.325718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:30.478767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:30.488916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a2ce5b7a-dadf-41e9-a520-22b99561c770 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.176495 Sep 19 16:47:30.489506 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1073/1928] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:30 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:31.512662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:31.513281 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:31.520071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:31.520558 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:31.520987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:31.521690 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:31.522140 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:31.522595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:31.654331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:31.663484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3e98f7d6-75d9-4ac6-b151-a404bd89f4db tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154985 Sep 19 16:47:31.664072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 856/1929] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:31 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:32.687647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:32.688168 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:32.690509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:32.695870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:32.696348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:32.696783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:32.697471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:32.697925 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:32.698353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:32.832206 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:32.840476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38ac1605-4a89-4225-96c2-f51808b1e8f7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157291 Sep 19 16:47:32.841053 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1074/1930] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:32 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:33.864147 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:33.864772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:33.871370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:33.871881 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:33.872323 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:33.873025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:33.873485 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:33.873947 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:34.012385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:34.023297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-601c5337-1008-4f55-8b3a-314acc362aa8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.163442 Sep 19 16:47:34.023878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 857/1931] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:35.046404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:35.046979 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:35.053956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:35.054474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:35.054913 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:35.055603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:35.056307 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:35.056736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:35.192457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:35.201161 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3f54c806-624f-455f-bf02-932bfcef0fb5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158756 Sep 19 16:47:35.201735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1075/1932] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:36.225575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:36.226192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:36.232992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:36.233469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:36.234519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:36.235227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:36.235684 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:36.236108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:36.371181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:36.380160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f8bc5f00-2d6c-486d-b30a-a063ed87da3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158020 Sep 19 16:47:36.386539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 858/1933] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:36 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:37.403247 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:37.403874 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:37.411514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:37.412054 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:37.412477 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:37.413160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:37.414046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:37.414046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:37.589319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:37.600246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e77ba593-6168-4f52-a463-ffe07e0b19c9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.200774 Sep 19 16:47:37.600842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1076/1934] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:38.623845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:38.625016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:38.632263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:38.632746 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:38.633177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:38.633869 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:38.634309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:38.634753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:39.017023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:39.026300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f0ec44e8-6cac-48ec-9ff4-56ceb9a53c37 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.406159 Sep 19 16:47:39.026873 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 859/1935] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:38 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:40.053796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:40.054428 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:40.060886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:40.061738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:40.062339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:40.062991 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:40.063437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:40.063862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:40.214013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:40.223520 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0372a254-dcd8-4f1f-8ffa-dfed399d2c7c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.174414 Sep 19 16:47:40.224112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1077/1936] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:40 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:41.246222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:41.246900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:41.253536 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:41.254011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:41.254464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:41.255159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:41.255629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:41.256045 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:41.420734 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:41.429803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6411c219-2ab5-4cdf-bcf3-c8ba2bdb9f0c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.187292 Sep 19 16:47:41.430432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 860/1937] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:42.457475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:42.458068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:42.464709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:42.465185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:42.465613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:42.466314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:42.466784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:42.467201 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:42.608234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:42.616626 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a309af19-ddc2-4d28-b5fc-441c829acbf9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.162943 Sep 19 16:47:42.617308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1078/1938] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:43.640717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:43.641328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:43.648672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:43.649182 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:43.649613 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:43.650328 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:43.650801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:43.651221 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:43.653890 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:43.787516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:43.796008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b73b68a4-dab0-4662-9bec-43ea663b95aa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159286 Sep 19 16:47:43.797161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 861/1939] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:43 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:44.819753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:44.820368 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:44.827007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:44.827498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:44.827942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:44.828679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:44.829106 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:44.829532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:44.966849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:44.975405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4037af2f-a25d-4f86-b893-a8fe4075b533 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159488 Sep 19 16:47:44.975984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1079/1940] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:44 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:45.999717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:46.000259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:46.006663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:46.007142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:46.007580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:46.008274 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:46.008721 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:46.009144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:46.141390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:46.150046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ae92166f-04bd-4a39-bab3-2f2272ccc012 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154625 Sep 19 16:47:46.150723 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 862/1941] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:45 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:47.173523 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:47.174129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:47.181079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:47.181585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:47.182066 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:47.182800 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:47.183246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:47.183663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:47.320887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:47.332056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b4d01d6f-d2cb-4897-8299-b97338598c1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161941 Sep 19 16:47:47.332292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1080/1942] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:48.355624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:48.356241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:48.362827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:48.363311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:48.363749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:48.364446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:48.364895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:48.365321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:48.495842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:48.503950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad10afbd-4b08-4456-b7ed-37e8dfd34f5f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152337 Sep 19 16:47:48.504535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 863/1943] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:47:49.526787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:49.527397 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:49.533744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:49.534209 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:49.534689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:49.535366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:49.535804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:49.536221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:49.538791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:49.675210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:49.683701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-118c9e39-3f09-4dad-968c-1588e53d2903 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160903 Sep 19 16:47:49.684276 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1081/1944] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:50.708819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:50.709568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:50.716114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:50.716668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:50.717104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:50.718284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:50.718757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:50.719202 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:50.850114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:50.858351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c47d92a1-5c8d-4a54-b6d2-c84af0fc3d8b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153300 Sep 19 16:47:50.859027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 864/1945] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:50 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:51.881324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:51.881970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:51.888521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:51.889002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:51.889440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:51.890140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:51.890627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:51.891055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:52.022411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:52.031243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ee918b00-173a-47d2-8237-8a19eac80019 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153721 Sep 19 16:47:52.031837 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1082/1946] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:51 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:47:53.055209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:53.056025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:53.062682 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:53.063164 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:53.064590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:53.064891 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:53.065459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:53.065762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:53.200085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:53.208608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-47405db7-49b7-4082-b0a7-c20fd9b95816 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157262 Sep 19 16:47:53.209186 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 865/1947] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:53 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:54.233210 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:54.233811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:54.240724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:54.241212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:54.241655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:54.242432 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:54.242893 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:54.243321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:54.402714 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:54.417086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5c010aa8-8756-4dd8-9129-561c36b61614 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.187363 Sep 19 16:47:54.417965 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1083/1948] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:55.440895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:55.441513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:55.443866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:47:55.449516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:55.449996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:55.450464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:55.451153 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:55.451608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:55.452034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:55.582956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:55.591735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ec4f501f-e7af-4837-8ba1-cee6dff5c7fc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154533 Sep 19 16:47:55.592567 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 866/1949] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:56.616920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:56.617692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:56.624548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:56.625032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:56.625469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:56.626169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:56.626655 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:56.627084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:56.767284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:56.775847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-88c10557-6a2a-469b-babd-35bc1a59deaa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.163075 Sep 19 16:47:56.776429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1084/1950] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:47:57.800038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:57.800655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:57.811507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:57.811990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:57.812431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:57.813135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:57.813588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:57.814014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:57.940235 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:57.948694 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e49ae85a-abb7-4047-8a4d-dd1a2ec5ab43 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152488 Sep 19 16:47:57.949264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 867/1951] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:57 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:47:58.973901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:47:58.974546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:47:58.985956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:58.986472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:58.986919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:58.987614 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:47:58.988069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:47:58.988494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:47:59.123617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:47:59.131774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9bf9e084-bcef-47e9-b341-11fa553c6885 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.162573 Sep 19 16:47:59.132350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1085/1952] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:47:58 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:00.155492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:00.156051 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:00.163010 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:00.163491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:00.163934 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:00.164626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:00.165079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:00.165507 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:00.298870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:00.307388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-39b8def6-900e-4dc6-972c-d68d60af0078 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155852 Sep 19 16:48:00.307971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 868/1953] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:00 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:48:01.333823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:01.334597 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:01.341296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:01.341762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:01.342186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:01.343004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:01.343489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:01.344003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:01.472219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:01.480424 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1d4d080b-7c66-45b6-8e31-a1ef8a11b697 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152035 Sep 19 16:48:01.480990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1086/1954] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:02.503835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:02.504451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:02.511052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:02.511532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:02.511971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:02.512663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:02.513384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:02.513819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:02.643041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:02.651453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c812cf26-bfb5-4543-8dac-4ccfc2029eae tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151781 Sep 19 16:48:02.652015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 869/1955] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:02 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:03.676176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:03.676769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:03.679079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:03.685097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:03.685604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:03.686033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:03.686741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:03.687185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:03.687602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:03.823705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:03.832087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f7513dbb-acbc-41ab-a297-5a30af9f4ff1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160791 Sep 19 16:48:03.832660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1087/1956] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:04.856559 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:04.857173 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:04.863770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:04.864244 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:04.864672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:04.865351 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:04.865773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:04.866252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:04.996220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:05.006843 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-da24d2a7-5283-4ee6-b678-f500a4578f49 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153681 Sep 19 16:48:05.007903 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 870/1957] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:04 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:06.038393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:06.039019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:06.045557 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:06.046034 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:06.046505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:06.047197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:06.047645 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:06.048064 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:06.182882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:06.192060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e656322-bd86-448f-873d-0b807d6440a2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157495 Sep 19 16:48:06.192625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1088/1958] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:06 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:07.216215 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:07.216866 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:07.224044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:07.224523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:07.224957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:07.225652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:07.226101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:07.226554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:07.362392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:07.370571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ef717dd-30a6-4b0d-9f1f-eb64d31ce6f1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158498 Sep 19 16:48:07.371150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 871/1959] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:07 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:08.394068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:08.394719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:08.401048 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:08.401528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:08.401967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:08.402685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:08.403140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:08.403567 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:08.535278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:08.543581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eb7ca605-a861-4605-8585-95fdbdbd44e5 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153346 Sep 19 16:48:08.544160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1089/1960] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:08 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:09.568858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:09.569475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:09.575925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:09.576477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:09.576837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:09.577520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:09.577986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:09.578469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:09.703271 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:09.711392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e90d63be-9943-418b-b782-6c98cf70bacd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.147043 Sep 19 16:48:09.711969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 872/1961] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:09 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:48:10.734543 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:10.735177 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:10.741654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:10.742143 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:10.742618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:10.743306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:10.743760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:10.744187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:10.879770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:10.888346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1d820b4f-2abf-435b-8362-e8b12801919c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157666 Sep 19 16:48:10.888900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1090/1962] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:10 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:11.912629 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:11.913384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:11.920027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:11.920505 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:11.920944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:11.921640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:11.922092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:11.922553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:12.059125 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:12.067840 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b957e33e-b965-48bb-9731-73f857c5b63b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159078 Sep 19 16:48:12.068402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 873/1963] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:11 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:13.093961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:13.094849 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:13.101879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:13.102355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:13.102817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:13.103505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:13.103947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:13.104366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:13.387039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:13.395789 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-598588c4-0313-4811-a9ea-2fb15b7fbbf8 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.307543 Sep 19 16:48:13.396370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1091/1964] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:13 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 309 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:14.418813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:14.419435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:14.426775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:14.427180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:14.427621 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:14.428312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:14.428768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:14.429200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:14.433405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:14.562192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:14.570516 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3b1b5be2-6ba9-4213-9705-ecc690f02e60 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155687 Sep 19 16:48:14.571082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 874/1965] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:14 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:15.593314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:15.593926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:15.600306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:15.600767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:15.601199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:15.601916 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:15.602363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:15.602826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:15.739566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:15.747707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c61cfb8-b68f-4b87-ba42-85eccaf95f8c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158213 Sep 19 16:48:15.748262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1092/1966] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:15 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:16.772175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:16.772895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:16.779419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:16.779789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:16.780229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:16.780929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:16.781378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:16.781800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:16.912386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:16.920496 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d57d44bf-4a6a-4ebf-9a40-8d57c8949219 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152211 Sep 19 16:48:16.921052 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 875/1967] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:16 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:17.944371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:17.944984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:17.951451 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:17.951923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:17.952353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:17.953040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:17.953486 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:17.953903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:18.082867 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:18.091542 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8c2a67ef-8ae3-4def-9343-0f5bfd5df12e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151014 Sep 19 16:48:18.092129 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1093/1968] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:17 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:19.115883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:19.116498 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:19.122482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:19.122960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:19.123392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:19.124079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:19.124527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:19.124955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:19.252939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:19.260963 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5fb69fd5-940a-43d6-b5cc-ee20cbb92b02 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149121 Sep 19 16:48:19.261522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 876/1969] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:19 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:20.285731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:20.286301 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:20.292697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:20.293175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:20.293610 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:20.294302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:20.294787 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:20.295217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:20.297728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:20.439532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:20.447646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-289a4aa6-0d5a-469e-ad16-77ee0e9f6d71 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.165967 Sep 19 16:48:20.448223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1094/1970] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:20 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:21.471308 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:21.471953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:21.478722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:21.479422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:21.479857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:21.480557 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:21.481102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:21.481421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:21.616191 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:21.624243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ae493c1-709b-49a4-a62e-23a24bbbcd5a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156857 Sep 19 16:48:21.624800 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 877/1971] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:21 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:22.647792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:22.648405 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:22.654966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:22.655445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:22.655879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:22.656569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:22.657025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:22.657452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:22.789636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:22.798111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-105fc4d7-5c35-44e1-8a70-fda1fc5871da tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154207 Sep 19 16:48:22.798738 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1095/1972] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:22 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:23.822070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:23.822640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:23.829573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:23.830058 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:23.830525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:23.831212 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:23.831662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:23.832083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:23.970852 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:23.979293 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-180d0cce-a0c3-493f-aa2e-3dd36e3d6a20 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.161185 Sep 19 16:48:23.980024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 878/1973] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:23 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:25.001654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:25.002299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:25.008673 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:25.009149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:25.009582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:25.010268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:25.010747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:25.011171 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:25.140962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:25.148974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-72dfe331-6fb0-43eb-952f-b2c1df2630e9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.151167 Sep 19 16:48:25.149561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1096/1974] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:24 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:26.172361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:26.173474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:26.175861 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:26.181317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:26.181785 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:26.182216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:26.182924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:26.183365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:26.183780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:26.316219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:26.324745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b07b196d-1f32-4a3e-a6dc-c5d5d57a9f0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156156 Sep 19 16:48:26.325313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 879/1975] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:26 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:27.347590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:27.348202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:27.354596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:27.355074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:27.355504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:27.356193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:27.356644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:27.357067 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:27.494004 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:27.502689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e7064bcb-fd6d-4f42-9aed-74ffe07d1b2a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159009 Sep 19 16:48:27.503258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1097/1976] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:27 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:28.525680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:28.526296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:28.533395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:28.533875 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:28.534312 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:28.535026 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:28.535482 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:28.535911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:28.667711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:28.675936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fa487afe-af17-4f40-95e1-112bf104f70e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154149 Sep 19 16:48:28.676509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 880/1977] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:28 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:29.700568 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:29.701098 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:29.707258 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:29.707741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:29.708178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:29.708864 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:29.709316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:29.709741 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:29.842953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:29.851224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4520cffd-676b-4fdf-8984-bf71b178cc1c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154656 Sep 19 16:48:29.851806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1098/1978] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:29 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:30.875315 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:30.875931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:30.882922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:30.883397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:30.883835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:30.884518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:30.884969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:30.885397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:31.013628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:31.025267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-34439554-6f9b-4d0e-81d6-c9b6bda12886 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153765 Sep 19 16:48:31.025839 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 881/1979] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:30 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:32.052515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:32.053192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:32.059903 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:32.060395 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:32.060832 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:32.061529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:32.062047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:32.062547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:32.194344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:32.202550 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cbb1d197-c32f-49e1-a7d9-9623c0c8744c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157596 Sep 19 16:48:32.203119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1099/1980] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:32 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:33.225087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:33.225703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:33.232322 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:33.232809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:33.233249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:33.233948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:33.234436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:33.234908 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:33.367806 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:33.376214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-775b1a02-0650-4dbd-839b-fbdfe55b985f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154922 Sep 19 16:48:33.376865 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 882/1981] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:33 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:34.399707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:34.400328 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:34.402740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:34.408229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:34.408699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:34.409130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:34.409818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:34.410260 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:34.410705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:34.546641 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:34.555336 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d3184c6f-e6ed-4c79-ac89-1390c197e1f0 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160033 Sep 19 16:48:34.555901 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1100/1982] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:34 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:35.578602 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:35.579224 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:35.585639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:35.586134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:35.586595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:35.587297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:35.587751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:35.588361 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:35.717266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:35.725401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ca8f614-e459-4102-96f3-ff2e04f365eb tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150831 Sep 19 16:48:35.725954 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 883/1983] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:35 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:36.749091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:36.749709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:36.756191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:36.757162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:36.757593 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:36.758316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:36.758799 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:36.759228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:36.894007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:36.903060 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb8ccc9e-c824-4de7-ac50-723eb02f2c5d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157800 Sep 19 16:48:36.903722 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1101/1984] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:36 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:37.928207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:37.928967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:37.935662 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:37.936131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:37.936764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:37.937437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:37.937880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:37.938296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:38.066960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:38.075064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ab3bed40-544d-417a-a36e-d538638a61fd tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150604 Sep 19 16:48:38.075748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 884/1985] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:37 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:39.099253 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:39.099862 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:39.106918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:39.107388 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:39.107816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:39.108492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:39.108939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:39.109357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:39.239633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:39.248563 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2c24c018-7ae8-4b68-92c9-1980d4accf03 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153200 Sep 19 16:48:39.249122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1102/1986] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:39 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:40.273685 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:40.274280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:40.280858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:40.281335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:40.281767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:40.282495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:40.282945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:40.283367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:40.414546 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:40.422588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-27c541ca-791f-4a24-a38b-c2aeb8e885c6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153257 Sep 19 16:48:40.423150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 885/1987] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:40 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:41.447959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:41.448524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:41.455705 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:41.456183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:41.456619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:41.457316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:41.457772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:41.458198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:41.598548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:41.607089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-00b253bf-5d3c-42b0-bcb0-42cf3a3df396 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.163550 Sep 19 16:48:41.607746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1103/1988] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:41 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:42.630399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:42.631056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:42.637438 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:42.638048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:42.638532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:42.639245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:42.639699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:42.640124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:42.775452 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:42.783896 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-be0821bb-9009-41d0-9599-c56a1e154bf2 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157419 Sep 19 16:48:42.784469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 886/1989] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:42 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:43.808199 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:43.808823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:43.815305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:43.815786 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:43.816216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:43.816905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:43.817357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:43.817784 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:43.949290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:43.957853 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cd373359-6958-4944-baa6-03788eb46e3c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154418 Sep 19 16:48:43.958453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1104/1990] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:43 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:44.982535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:44.983187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:44.989644 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:44.990129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:44.990600 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:44.991618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:44.992870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:44.993310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:44.997374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=78 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:45.126899 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:45.135717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3ecba355-d7b3-4f84-9c9e-764236473069 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157312 Sep 19 16:48:45.136481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 887/1991] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:44 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:46.159120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:46.159740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:46.161690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:48:46.161845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:48:46 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:48:46.168382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:46.168856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:46.169293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:46.169980 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:46.170461 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:46.170886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:46.303752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:46.312659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-21dd8a94-14e3-46b3-ad12-d4daf67ed029 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.157342 Sep 19 16:48:46.313222 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1105/1992] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:46 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:48:47.337064 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:47.337616 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:47.343752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:47.344233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:47.344689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:47.345364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:47.345817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:47.346241 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:47.477420 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:47.489108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-760e3f00-84b3-417f-b200-b57f1e07f9b7 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156729 Sep 19 16:48:47.489688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 888/1993] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:47 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:48.512653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:48.513298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:48.519713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:48.520186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:48.520619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:48.521298 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:48.521744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:48.522162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:48.650254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:48.658519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-59e3112e-5b50-4763-af3d-1ffb1b350baa tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.149591 Sep 19 16:48:48.659094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1106/1994] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:48 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:49.681951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:49.682499 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:49.689300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:49.689793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:49.690206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:49.690921 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:49.691372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:49.691798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:49.826157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:49.834259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-faff5e3e-509a-41a1-85aa-a7c41c9b3201 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156492 Sep 19 16:48:49.834857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 889/1995] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:49 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:50.857978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:50.858621 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:50.864993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:50.865466 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:50.865898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:50.866676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:50.867061 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:50.867485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:50.870032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=78 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:51.005909 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:51.014539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-79051c17-1d71-4c99-affe-b25e08b0b7af tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.160368 Sep 19 16:48:51.015105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1107/1996] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:50 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:52.038544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:52.039157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:52.046252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:52.046754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:52.047184 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:52.047870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:52.048321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:52.048736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:52.345793 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:52.354163 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8b65b1ca-19ae-42e2-8add-031811850276 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.319557 Sep 19 16:48:52.354951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 890/1997] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:52 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:53.377970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:53.378613 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:53.385703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:53.386188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:53.386659 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:53.387344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:53.387825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:53.388226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:53.519571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:53.528308 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-96f9bb28-abca-4e0f-b462-db10779dfad1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.154306 Sep 19 16:48:53.528858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1108/1998] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:53 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:54.551518 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:54.552138 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:54.558836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:54.559310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:54.559749 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:54.560625 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:54.560917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:54.561342 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:54.692476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:54.701152 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-091a49cd-7d20-414d-a58b-8435997beb0d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.153478 Sep 19 16:48:54.701740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 891/1999] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:54 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:55.725055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:55.725675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:55.732639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:55.733115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:55.733551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:55.734239 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:55.734753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:55.735180 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:55.868657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:55.876691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8a9abbd1-4d53-4001-b48b-0b815e52782f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.155537 Sep 19 16:48:55.877269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1109/2000] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:55 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:56.900087 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:56.900701 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:56.904148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=26 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:48:56.909573 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:56.910000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:56.910467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:56.911196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:56.911672 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:56.912101 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:57.044693 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:57.052991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5afd076f-3efc-42ad-b116-c2ecbc790e70 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.156996 Sep 19 16:48:57.053561 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 892/2001] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:56 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:58.077151 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:58.077681 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:58.079682 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:48:58.079809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:48:58 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f (ip 10.222.0.57) !!! Sep 19 16:48:58.088943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:58.089468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:58.090035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:58.090900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:58.091449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:58.092001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:58.223752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:58.232085 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-be609eb4-a9ab-40a7-8763-3f7bd0df005f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.158785 Sep 19 16:48:58.232622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1110/2002] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:58 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:48:59.256127 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:48:59.256699 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:48:59.263989 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:59.264370 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:59.264811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:59.265517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:48:59.265981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:48:59.266432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:48:59.402517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:48:59.410883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0ebbfb4b-f6ad-4405-b1d8-ec21d0a0dc35 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.159009 Sep 19 16:48:59.411455 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 893/2003] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:48:59 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:00.434792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:00.435353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:00.441221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:00.441692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:00.442119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:00.442824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:00.443269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:00.443683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:00.572998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:00.581118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-53d2d11c-5a63-476c-9722-8125c222d182 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.150718 Sep 19 16:49:00.581675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1111/2004] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:00 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:01.605108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:01.605717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:01.612374 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:01.612853 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:01.613286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:01.613978 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:01.614459 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:01.614884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:01.744656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:01.753272 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bf528714-8b9e-4305-9544-43d003810a99 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.152071 Sep 19 16:49:01.764634 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 894/2005] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:01 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:02.779219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:02.779219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:02.785651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:02.786417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:02.786960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:02.787814 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:02.788498 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:02.789604 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:02.956103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:02.965584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a663f48b-7d40-4e37-8638-844874936656 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.191768 Sep 19 16:49:02.974878 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1112/2006] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:02 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:03.993904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:03.999782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:04.013940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.015082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.015520 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.016345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.017113 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.017650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.073887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:04.074505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:04.081274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.082022 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.082706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.083616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.084069 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.084495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.118469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:49:04.195312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR oslo.messaging._drivers.impl_rabbit [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [195a8112-aa2d-4313-89c4-006d60901d3e] AMQP server on 10.222.0.57:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 19 16:49:04.212563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:04.224887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7565d5f6-bbfe-4caf-9f93-a85107de346a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1559 microversion: 2.1 time: 0.235768 Sep 19 16:49:04.225441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 895/2007] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:03 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1559 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:04.291820 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:04.292434 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:04.299901 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.300519 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.301022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.303311 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.303784 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.304218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.318503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 HTTP/1.1 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:04.320220 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.322006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'bae51cb6-fc0c-4343-9cd7-c04ef65d6b93'} Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:04.322937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.324544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.326092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:04.326092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e1c62b6b-dc64-4963-baa9-69e0c10b33bf tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 404 len: 111 microversion: 2.1 time: 0.035958 Sep 19 16:49:04.326092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 896/2008] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:49:04 2022] DELETE /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:04.330069 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:04.330757 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] [instance: bae51cb6-fc0c-4343-9cd7-c04ef65d6b93] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:04.337316 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.337858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.338286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.339063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.339521 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.340070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.357608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] HTTP exception thrown: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Request method failure captured: Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 HTTP/1.1 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU3D_AhIF-9jMGZy_222U37yP2J5iWSABT5eouj3v6Wh9ve26_KP6iohmJ6il48oB-eyQaA-nianlYjjPaYl_-COZsHimrKkxRjtKjlFzEElK5ksxqbXXlP8d0DpgQ6jWpl-8LkjcvfqFrxXlECgGVQVezcjwEitS2K8WrSOw6_Ynk Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:04.359553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/23c705bc001e4228bdfcdfa916a368e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_23c705bc001e4228bdfcdfa916a368e9"}]}] Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 23c705bc001e4228bdfcdfa916a368e9 Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeTestJSON-1157194210 Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 754c1b5cb47746cf93c52d6dc0b8fa01 Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeTestJSON-1157194210-project Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.361277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'bae51cb6-fc0c-4343-9cd7-c04ef65d6b93'} Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:04.362167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. Sep 19 16:49:04.363648 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:04.364941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] Returning 404 to user: Instance bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:04.364941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-959e2064-b907-4397-affc-80089aa9656a tempest-AttachVolumeTestJSON-1157194210 tempest-AttachVolumeTestJSON-1157194210-project] 10.222.0.57 "GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93" status: 404 len: 111 microversion: 2.1 time: 0.034840 Sep 19 16:49:04.364941 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 897/2009] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:04 2022] GET /compute/v2.1/servers/bae51cb6-fc0c-4343-9cd7-c04ef65d6b93 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:04.721041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:04.721571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:04.728041 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.728541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.728985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.729707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:04.730160 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:04.730620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:04.753503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lazy-loading 'flavor' on Instance uuid 90999c63-0653-4625-85c8-9ee565c5d780 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:04.951802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR oslo.messaging._drivers.impl_rabbit [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [18a24371-6ef5-4b8a-bfa4-d68cbb899d17] AMQP server on 10.222.0.57:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 19 16:49:05.204872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [195a8112-aa2d-4313-89c4-006d60901d3e] Reconnected to AMQP server on 10.222.0.57:5672 via [amqp] client with port 56498. Sep 19 16:49:05.217266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=28 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:05.242299 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-37cdd0da-0232-4f8f-b99e-edcb35577aff tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 204 len: 0 microversion: 2.1 time: 1.169989 Sep 19 16:49:05.245093 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1113/2010] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:49:04 2022] DELETE /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 0 bytes in 1176 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:49:05.254501 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:05.255240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:05.264019 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:05.264514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:05.265008 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:05.265961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:05.266605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:05.267255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:05.741580 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:05.752899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-d02b8a9f-b8c1-4642-af61-88325e18b511 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1513 microversion: 2.1 time: 0.506395 Sep 19 16:49:05.768367 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1114/2011] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:05 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1513 bytes in 522 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:05.776713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:05.778960 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:05.794056 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:05.796028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:05.796907 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:05.797806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:05.798643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:05.799627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:05.963018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [18a24371-6ef5-4b8a-bfa4-d68cbb899d17] Reconnected to AMQP server on 10.222.0.57:5672 via [amqp] client with port 56514. Sep 19 16:49:05.967585 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-676e7db9-0312-48d5-8d35-cd4da146ac0b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/action" status: 202 len: 0 microversion: 2.20 time: 1.250597 Sep 19 16:49:05.969245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 898/2012] 10.222.0.57 () {62 vars in 1319 bytes} [Mon Sep 19 16:49:04 2022] POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/action => generated 0 bytes in 1253 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:49:06.108487 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:06.111106 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f", "name": "network-vif-deleted", "tag": "2f27510e-988b-4adf-8776-39c67b3541ce"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:06.118917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e5a73db1-da1d-4335-b635-16942b1e07a0 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1689 microversion: 2.1 time: 0.349322 Sep 19 16:49:06.119586 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1115/2013] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:05 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1689 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:06.129275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:06.130035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:06.141510 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.142257 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.143128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.144148 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.144636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.145029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.145708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.146596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.146741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.164348 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] Creating event network-vif-deleted:2f27510e-988b-4adf-8776-39c67b3541ce for instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f on hv-cinder-85837 Sep 19 16:49:06.169581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-8cd4f2aa-ead6-4982-9a0e-57f0f8918e03 req-7ec24609-c511-44f9-8a58-f2f72b638829 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.198728 Sep 19 16:49:06.170993 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 899/2014] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:06.666080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:06.696519 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:06.701319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:06.704227 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f470aaee-9c6a-4aeb-a0e3-1bafba393a1f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.583605 Sep 19 16:49:06.705879 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1116/2015] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:06 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 586 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:06.783034 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:06.783814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:06.793715 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.796365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.802199 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.803242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.804522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.809422 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.929486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:06.930007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:06.933486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:06.935642 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'fault' on Instance uuid ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:07.061099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:07.074704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b8cc1bce-4b02-4ca2-975c-1b64a9107a27 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 200 len: 1308 microversion: 2.1 time: 0.294779 Sep 19 16:49:07.077549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 900/2016] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:06 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 1308 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:07.143398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:07.143977 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:07.156188 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:07.156813 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:07.157718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:07.158988 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:07.160311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:07.162343 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:07.216873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:07.217773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:07.219516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:07.220152 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'fault' on Instance uuid fbfe137a-fc97-409e-9473-71a345427ae2 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:07.364172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:07.373616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e10156ae-d00c-4354-8250-6999e3c0672e tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 200 len: 1335 microversion: 2.1 time: 0.233099 Sep 19 16:49:07.374508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1117/2017] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:07 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 1335 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:07.736402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:07.737385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:07.747097 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:07.748363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:07.749900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:07.751809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:07.752483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:07.754442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.023762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:08.052157 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:08.055128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:08.056897 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0057749c-5a99-4313-bc2e-a48fc717d9b3 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.325818 Sep 19 16:49:08.057686 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 901/2018] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:07 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:08.105353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:08.106345 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:08.119531 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:08.120314 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:08.120691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.127025 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:08.127687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:08.128244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.156194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f HTTP/1.1 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:08.157888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. Sep 19 16:49:08.159734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f'} Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:08.160739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. Sep 19 16:49:08.162650 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.164189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 404 to user: Instance ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:08.164189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-846929ab-0635-42f5-abc5-70d4c5641e95 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f" status: 404 len: 111 microversion: 2.1 time: 0.059815 Sep 19 16:49:08.164189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1118/2019] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:08 2022] GET /compute/v2.1/servers/ed74bcad-817c-40ca-8bfb-23b8d1e5ea4f => generated 111 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:08.270408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fbfe137a-fc97-409e-9473-71a345427ae2", "name": "network-vif-deleted", "tag": "d08233fe-cd43-4b70-9e20-28e916f15db6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:08.281838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:08.282618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:08.283363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.318705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] Creating event network-vif-deleted:d08233fe-cd43-4b70-9e20-28e916f15db6 for instance fbfe137a-fc97-409e-9473-71a345427ae2 on hv-cinder-85837 Sep 19 16:49:08.346266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-cdbec404-104a-4424-b5ce-de2bfbc3d283 req-b6b6ec4f-6873-4cf5-a3c7-73edfb9bdb1c service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.083037 Sep 19 16:49:08.347504 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 902/2020] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 86 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:08.397533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:08.398140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: fbfe137a-fc97-409e-9473-71a345427ae2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:08.410751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:08.410751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:08.410751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.410751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:08.412688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:08.414595 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:08.438947 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 HTTP/1.1 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:08.440596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. Sep 19 16:49:08.442353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'fbfe137a-fc97-409e-9473-71a345427ae2'} Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:08.443458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. Sep 19 16:49:08.445664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:08.448063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 404 to user: Instance fbfe137a-fc97-409e-9473-71a345427ae2 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:08.448063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb77f635-59df-4809-83eb-31fbca905859 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2" status: 404 len: 111 microversion: 2.1 time: 0.049608 Sep 19 16:49:08.448063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1119/2021] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:08 2022] GET /compute/v2.1/servers/fbfe137a-fc97-409e-9473-71a345427ae2 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:49:08.474190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e61a92c5-6c98-47e7-8b5c-da18392c9c47 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '.version_select at 0x7fadc3037040>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:08.534999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e61a92c5-6c98-47e7-8b5c-da18392c9c47 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2029727403" status: 202 len: 0 microversion: 2.1 time: 0.083505 Sep 19 16:49:08.537668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 903/2022] 10.222.0.57 () {58 vars in 1221 bytes} [Mon Sep 19 16:49:08 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2029727403 => generated 0 bytes in 89 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:49:09.084278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:09.084820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:09.091546 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:09.092290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:09.092894 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:09.093685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:09.094406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:09.094996 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:09.439173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:09.448040 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:09.449668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:09.451140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-73d69766-216f-4172-8bf4-010f67ee45fe tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.371605 Sep 19 16:49:09.451689 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1120/2023] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:09 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:10.479907 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:10.481282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:10.488110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:10.488845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:10.489291 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:10.490177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:10.490664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:10.492445 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:10.699144 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:10.708300 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:10.709766 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:10.711055 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b514b4c6-b971-44ba-a2a6-d12766b6d58c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.237133 Sep 19 16:49:10.711610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 904/2024] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:10 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:10.955703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e2b0e748-fcd4-400e-a677-53525a765884 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Action: 'create', calling method: .version_select at 0x7fadc31148b0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1522529206", "metadata": {"Type": "work"}}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e2b0e748-fcd4-400e-a677-53525a765884 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 122 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:10.964667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1522529206", "metadata": {"Type": "work"}}} Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:49:10.966295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1522529206', 'metadata': {'Type': 'work'}}}} Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise error Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi '#$%' Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:10.967375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 19 16:49:10.971421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 19 16:49:10.974529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 19 16:49:10.974529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.974529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e2b0e748-fcd4-400e-a677-53525a765884 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:10.974529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e2b0e748-fcd4-400e-a677-53525a765884 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.139031 Sep 19 16:49:10.974529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1121/2025] 10.222.0.57 () {60 vars in 1156 bytes} [Mon Sep 19 16:49:10 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 140 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:49:10.977001 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-25e2c023-9bca-4c77-aa82-92d493ceee8e tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Action: 'create', calling method: .version_select at 0x7fadc3157160>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2094589362", "metadata": {"Type": "work"}}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-25e2c023-9bca-4c77-aa82-92d493ceee8e tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 120 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:10.982778 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-2094589362", "metadata": {"Type": "work"}}} Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:49:10.985167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-2094589362', 'metadata': {'Type': 'work'}}}} Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise error Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi '0' Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:10.986290 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 19 16:49:10.988279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 19 16:49:10.989837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 19 16:49:10.989837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:10.992046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-25e2c023-9bca-4c77-aa82-92d493ceee8e tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:10.993151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-25e2c023-9bca-4c77-aa82-92d493ceee8e tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.020738 Sep 19 16:49:10.993902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 905/2026] 10.222.0.57 () {60 vars in 1156 bytes} [Mon Sep 19 16:49:10 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 22 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:49:11.003570 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a234d9e2-141d-40d5-b03b-128434bdade7 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Action: 'create', calling method: .version_select at 0x7fadbf6ed790>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-749576525", "metadata": {"Type": "work"}}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a234d9e2-141d-40d5-b03b-128434bdade7 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 118 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:11.009469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-749576525", "metadata": {"Type": "work"}}} Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:49:11.011561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-749576525', 'metadata': {'Type': 'work'}}}} Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise error Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi '' Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:11.012656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 19 16:49:11.014300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 19 16:49:11.015872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 19 16:49:11.015872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.015872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a234d9e2-141d-40d5-b03b-128434bdade7 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:11.015872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a234d9e2-141d-40d5-b03b-128434bdade7 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.011628 Sep 19 16:49:11.015872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1122/2027] 10.222.0.57 () {60 vars in 1156 bytes} [Mon Sep 19 16:49:10 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 13 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:49:11.023070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cbd04186-2612-4108-83b7-4db18bc02ed5 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Calling method '.version_select at 0x7fadbf83c0d0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:11.073704 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-cbd04186-2612-4108-83b7-4db18bc02ed5 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] HTTP exception thrown: Volume tempest-invalid-1936943371 could not be found. Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cbd04186-2612-4108-83b7-4db18bc02ed5 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-1936943371 HTTP/1.1 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:11.076027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Volume tempest-invalid-1936943371 could not be found. Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'tempest-invalid-1936943371'} Sep 19 16:49:11.077951 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-1936943371 could not be found. (HTTP 404) (Request-ID: req-03ec79a9-64f9-4b24-8441-45a427560d42) Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.079168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:49:11.080832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-1936943371 could not be found. Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:11.082335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-1936943371 could not be found. Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-1936943371 could not be found. Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-cbd04186-2612-4108-83b7-4db18bc02ed5 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 404 to user: Volume tempest-invalid-1936943371 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-cbd04186-2612-4108-83b7-4db18bc02ed5 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/tempest-invalid-1936943371" status: 404 len: 99 microversion: 2.1 time: 0.059228 Sep 19 16:49:11.084033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 906/2028] 10.222.0.57 () {58 vars in 1218 bytes} [Mon Sep 19 16:49:11 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-1936943371 => generated 99 bytes in 61 msecs (HTTP/1.1 404) 9 headers in 378 bytes (1 switches on core 0) Sep 19 16:49:11.088100 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-71352b95-82b7-400a-a92a-392e58ae7aa0 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004610 Sep 19 16:49:11.088591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1123/2029] 10.222.0.57 () {58 vars in 1140 bytes} [Mon Sep 19 16:49:11 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 19 16:49:11.097987 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6e66b774-f94f-4b12-8043-a3164032790a tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.003758 Sep 19 16:49:11.098580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 907/2030] 10.222.0.57 () {58 vars in 1137 bytes} [Mon Sep 19 16:49:11 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 5 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Sep 19 16:49:11.108612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa494c2b-b4c1-4a6e-ad5a-f2605e5fd485 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Calling method '.version_select at 0x7fadc3114c10>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:11.139091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-aa494c2b-b4c1-4a6e-ad5a-f2605e5fd485 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] HTTP exception thrown: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa494c2b-b4c1-4a6e-ad5a-f2605e5fd485 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/os-volumes/be0fecb1-0b61-4200-95b3-a0ed92c77187 HTTP/1.1 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:11.141332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'be0fecb1-0b61-4200-95b3-a0ed92c77187'} Sep 19 16:49:11.143118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. (HTTP 404) (Request-ID: req-0070fe02-de96-4cb9-a2ae-31ae29457e9c) Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.149631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:49:11.151920 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 19 16:49:11.153858 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-aa494c2b-b4c1-4a6e-ad5a-f2605e5fd485 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 404 to user: Volume be0fecb1-0b61-4200-95b3-a0ed92c77187 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-aa494c2b-b4c1-4a6e-ad5a-f2605e5fd485 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/be0fecb1-0b61-4200-95b3-a0ed92c77187" status: 404 len: 109 microversion: 2.1 time: 0.037967 Sep 19 16:49:11.155481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1124/2031] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:49:11 2022] DELETE /compute/v2.1/os-volumes/be0fecb1-0b61-4200-95b3-a0ed92c77187 => generated 109 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:49:11.156750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-05c8d25c-ec4c-477a-a341-23f095d03f11 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Calling method '.version_select at 0x7fadc3157700>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:11.180967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-05c8d25c-ec4c-477a-a341-23f095d03f11 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] HTTP exception thrown: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-05c8d25c-ec4c-477a-a341-23f095d03f11 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Request method failure captured: Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/os-volumes/7452c841-1a6c-417a-a69e-7074ad527756 HTTP/1.1 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ2GBVfb6N87CCj3cRNjhjKCTT2UYweroeOk9oKXK5yNWEQicJL34qMwI0nnUVEwqbvTwTmrjyyUPuHwDkBkip_Bvx6HTAkF-cnUuTAQzqxE6XXdnGX_IjaK_O-22ZIpKn4S_6gonkEuMc_tCU7rwMeWwhjfB3x0GgP2ULF8Vtx1wrU Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:11.183225 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b27fc770f9ce456ebfbf373a9836f3e9"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b27fc770f9ce456ebfbf373a9836f3e9"}]}] Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: b27fc770f9ce456ebfbf373a9836f3e9 Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesNegativeTest-492219563 Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 62767f55b93b4f469afa71a7d55ead34 Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesNegativeTest-492219563-project Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '7452c841-1a6c-417a-a69e-7074ad527756'} Sep 19 16:49:11.185018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. (HTTP 404) (Request-ID: req-aa73aab6-3860-40c2-8f77-e7d6fa839ec5) Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.186103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:49:11.187819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 19 16:49:11.189474 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-05c8d25c-ec4c-477a-a341-23f095d03f11 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] Returning 404 to user: Volume 7452c841-1a6c-417a-a69e-7074ad527756 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-05c8d25c-ec4c-477a-a341-23f095d03f11 tempest-VolumesNegativeTest-492219563 tempest-VolumesNegativeTest-492219563-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/7452c841-1a6c-417a-a69e-7074ad527756" status: 404 len: 109 microversion: 2.1 time: 0.035265 Sep 19 16:49:11.191270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 908/2032] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:49:11 2022] GET /compute/v2.1/os-volumes/7452c841-1a6c-417a-a69e-7074ad527756 => generated 109 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:49:11.739081 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:11.740015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:11.747331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:11.747826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:11.748474 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:11.749363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:11.749816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:11.750244 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:11.938654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:11.949091 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:11.950702 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:11.951993 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-96503e12-9839-4b70-9401-808d8ea120cb tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.217686 Sep 19 16:49:11.952701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1125/2033] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:11 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:12.545596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:12.546931 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:12.561511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:12.562099 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:12.562565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:12.563568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:12.564282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:12.564871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:12.569624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:12.599083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:49:12.715775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0e163e2b-0a34-4825-9464-9b82d7b40302 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 204 len: 0 microversion: 2.1 time: 0.176803 Sep 19 16:49:12.716739 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 909/2034] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:49:12 2022] DELETE /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:49:12.726804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:12.727539 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:12.734080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:12.734657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:12.735509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:12.736346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:12.736772 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:12.737288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:12.888112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:12.899007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f3283d27-9c35-4180-bae9-ea5eca1883b7 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1601 microversion: 2.1 time: 0.176972 Sep 19 16:49:12.900155 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1126/2035] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:12 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1601 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:12.980658 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:12.985765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:13.000735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:13.000735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:13.000735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:13.000735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:13.001570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:13.001570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:13.193096 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:13.205913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:13.207599 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:13.209057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1e194513-0652-4368-9c50-51d7b43d18e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.234154 Sep 19 16:49:13.210180 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 910/2036] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:12 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:13.924927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:13.925723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:13.936240 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:13.937219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:13.937961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:13.939065 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:13.939760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:13.941335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:13.946952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:14.017223 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:14.026306 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0de521b2-adda-492f-a77c-c91680af4b7c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1559 microversion: 2.1 time: 0.105337 Sep 19 16:49:14.039049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1127/2037] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:13 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1559 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:14.230395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4afae149-ed22-4a06-a4b0-d87d726d834f", "name": "network-vif-deleted", "tag": "96c941aa-fa1b-4d66-97da-985273910cfe"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:14.236110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:14.236793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:14.248521 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.250974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.252569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.253347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.253823 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.258632 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.265680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.266331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.266929 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.317534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] Creating event network-vif-deleted:96c941aa-fa1b-4d66-97da-985273910cfe for instance 4afae149-ed22-4a06-a4b0-d87d726d834f on hv-cinder-85837 Sep 19 16:49:14.319572 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:49:14.331230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-aa8db0bc-785a-4941-ae29-49e8919ba40f req-5fe33f3e-d48a-423c-b047-f2f3929a3868 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.103767 Sep 19 16:49:14.332608 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 911/2038] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:14.355124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:14.357627 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:14.374346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.374916 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.375392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.376259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.376745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.377181 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.535200 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4b5ed478-7aaa-427e-8e90-c5c2ac81743c tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 204 len: 0 microversion: 2.1 time: 0.293999 Sep 19 16:49:14.535803 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1128/2039] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:49:14 2022] DELETE /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:49:14.545978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:14.547001 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:14.553224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.553761 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.554191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.555086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:14.555544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:14.555968 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:14.608841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:14.622006 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:14.623872 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:14.625094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-14181599-a85d-4ffb-892e-fd86aae1329d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.288437 Sep 19 16:49:14.625649 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 912/2040] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:14 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:14.718928 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:14.728683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e97aecf0-3c73-44ba-b660-4af9682edf41 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1506 microversion: 2.1 time: 0.187896 Sep 19 16:49:14.729728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1129/2041] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:14 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1506 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:15.050607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:15.051227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:15.058074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.058624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.059085 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.059748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.060196 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.060624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.127902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:15.137302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-878c68ea-8842-4933-a255-58bde0d2cf1e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1390 microversion: 2.1 time: 0.090654 Sep 19 16:49:15.137880 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 913/2042] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:15 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1390 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:15.652411 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:15.653016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:15.659406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.659887 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.660327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.661021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.661473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.661900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.753713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:15.755218 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:15.781264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.783345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.788836 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.789046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:15.789711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:15.790145 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:15.808430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:15.818212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:15.820288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:15.821444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7633155d-b6c7-4333-b5e4-0ae2c1027403 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.173001 Sep 19 16:49:15.822007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1130/2043] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:15 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:15.925762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:15.933817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5ef94b3e-00c7-4f00-bc2f-9c669816bc0f tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1506 microversion: 2.1 time: 0.184790 Sep 19 16:49:15.934490 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 914/2044] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:15 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1506 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:16.160212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:16.160826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:16.167194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.167678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.168116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.168807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.169261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.169700 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.232216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:16.246186 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7865317f-791b-4677-8250-aabbe3f9542e tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1390 microversion: 2.1 time: 0.089341 Sep 19 16:49:16.247315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1131/2045] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:16 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1390 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:16.847692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:16.848318 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:16.855740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.856229 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.856670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.857382 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.857835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.858263 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.956890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:16.957500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:16.964402 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.964881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.965346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.966605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:16.966605 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:16.966910 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:16.991453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:16.999607 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:17.001054 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:17.002214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-50f7186a-810e-4dac-bbd9-209583266de0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.158595 Sep 19 16:49:17.002801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 915/2046] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:16 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:17.103779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:17.111709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c6edf63d-f7f6-4c02-aa5e-b30cb10058d1 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1506 microversion: 2.1 time: 0.158672 Sep 19 16:49:17.112281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1132/2047] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:16 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1506 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:49:17.270260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:17.270906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:17.278256 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:17.278772 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:17.279297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:17.280095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:17.280542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:17.281053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:17.349062 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:17.357472 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-62fd341b-72fa-4cb6-9c4f-a7b0fde5bd5c tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1390 microversion: 2.1 time: 0.091126 Sep 19 16:49:17.358046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 916/2048] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:17 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1390 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:18.035192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:18.035973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:18.045062 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.045551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.045973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.046723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.047532 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.047959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.131917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1836199861", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452", "networks": [{"uuid": "9949e5a2-e261-45a4-86fa-cce99f0752df"}]}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:18.230565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:18.233783 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] validate_networks() for [('9949e5a2-e261-45a4-86fa-cce99f0752df', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:49:18.239815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:18.241704 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:18.244162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9d0995c2-9471-4f43-ae14-bab7e755a00b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1745 microversion: 2.20 time: 0.216023 Sep 19 16:49:18.244914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1133/2049] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:18 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1745 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:18.250063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:18.250890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:18.257751 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.258490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.259122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.260028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.260647 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.261304 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.334112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:18.343842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cab9aabe-aa1d-4318-93ec-694ef4365bce tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 200 len: 1464 microversion: 2.1 time: 0.097728 Sep 19 16:49:18.345281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1134/2050] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:18 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 1464 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:18.350030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "7163ee7c-30ec-4aff-b4c6-5fe173a3efe2", "name": "network-vif-deleted", "tag": "4c4ec08a-9a03-4e16-91a0-f12747ea649c"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:18.360383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.360852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.361671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.374417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] Creating event network-vif-deleted:4c4ec08a-9a03-4e16-91a0-f12747ea649c for instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 on hv-cinder-85837 Sep 19 16:49:18.380283 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-f87e8341-e038-44a8-846a-0169e662dcc2 req-b087407c-f888-4022-b8e0-ce0d867255ef service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033614 Sep 19 16:49:18.381575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1135/2051] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:18 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:18.387125 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:18.387825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:18.394327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.397041 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.397656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.399207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.399828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.400423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.451845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.452606 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.453217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.453842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lazy-loading 'fault' on Instance uuid 4afae149-ed22-4a06-a4b0-d87d726d834f {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:18.509591 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '960704dfe18c4d739e90d00288724983', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '027b06570b2a41c4a0b43f3529340077', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:18.524790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bb8c2d9a-9e49-4fe7-8ea3-763a9ae92441 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 200 len: 1484 microversion: 2.1 time: 0.141723 Sep 19 16:49:18.528288 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1136/2052] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:18 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 1484 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:18.876501 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:49:18.876958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:49:18.877431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:49:18.877842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:49:18.878253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:49:18.887478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.887957 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.888395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.889119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:18.889571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:18.889997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:18.907189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Getting quotas for project 5fc6461a081c479fa20c9fb15a2b1575. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:49:18.911217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Getting quotas for user e1c0b2cfaa9f485f9116e012c62120fb and project 5fc6461a081c479fa20c9fb15a2b1575. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:49:18.922049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:49:18.924377 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:49:18.971895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b409780d-2f63-498b-8696-4d9025c0ac9a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.846479 Sep 19 16:49:18.972815 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 917/2053] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:49:18 2022] POST /compute/v2.1/servers => generated 380 bytes in 848 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:49:18.984393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-51ce561c-9c42-4ca5-bd7c-2d52b16dca8a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:18.985444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-51ce561c-9c42-4ca5-bd7c-2d52b16dca8a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.043584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-51ce561c-9c42-4ca5-bd7c-2d52b16dca8a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:19.052225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-51ce561c-9c42-4ca5-bd7c-2d52b16dca8a tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1219 microversion: 2.1 time: 0.071867 Sep 19 16:49:19.052808 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1137/2054] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:18 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1219 bytes in 73 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:19.270264 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.271086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.278415 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.278981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.279489 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.280253 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.280780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.281277 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.372144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.372951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.379218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.379708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.380146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.380835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.381284 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.381710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.396127 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 HTTP/1.1 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:49:19.397989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. Sep 19 16:49:19.399885 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '7163ee7c-30ec-4aff-b4c6-5fe173a3efe2'} Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:19.400886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. Sep 19 16:49:19.402798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.405956 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 404 to user: Instance 7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:19.407252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dfc48042-9982-49c6-93f5-0ec95250ba2a tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2" status: 404 len: 111 microversion: 2.1 time: 0.039686 Sep 19 16:49:19.408371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1138/2055] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:19 2022] GET /compute/v2.1/servers/7163ee7c-30ec-4aff-b4c6-5fe173a3efe2 => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:19.420434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.421007 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.421525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:19.427657 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.428148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.428633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.429352 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.430538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.430872 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.432949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:19.434813 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:19.436398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e04b769a-fb82-4851-b234-070df369a854 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1745 microversion: 2.20 time: 0.169993 Sep 19 16:49:19.438345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 918/2056] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:19 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1745 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:19.470815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:49:19.554400 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.556053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] [instance: 4afae149-ed22-4a06-a4b0-d87d726d834f] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.561195 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e7f29c8f-b7ce-487b-9fa6-fdf759059ded tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "DELETE /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 204 len: 0 microversion: 2.1 time: 0.147488 Sep 19 16:49:19.562618 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1139/2057] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:49:19 2022] DELETE /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 19 16:49:19.571604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.572188 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.572636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.573344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.573799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.574222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.576431 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.577739 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:19.588734 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.589535 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] HTTP exception thrown: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. Sep 19 16:49:19.589696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.590666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Request method failure captured: Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f HTTP/1.1 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJYJziU0iK9eoPQlJnnooQI51YWCH39nYGewXuokoRcoeg3TLegH1Dq8R4Xb4zSoN1gBzWfWH7co-HVgXMHEXqAvJ834STMA0Z9zqoI23K3pdBL5ojcFfkTKbgZg8K55hIOBjtL0Ca9NqfGrAW8BZnK7D_8wfaVrARDmMcuoTXif5lA Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:19.591811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/027b06570b2a41c4a0b43f3529340077"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_027b06570b2a41c4a0b43f3529340077"}]}] Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 027b06570b2a41c4a0b43f3529340077 Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-TestMinimumBasicScenario-220534008 Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 960704dfe18c4d739e90d00288724983 Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-TestMinimumBasicScenario-220534008-project Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. Sep 19 16:49:19.593448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '4afae149-ed22-4a06-a4b0-d87d726d834f'} Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:19.594331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. Sep 19 16:49:19.595775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:19.597576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:19.597683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Returning 404 to user: Instance 4afae149-ed22-4a06-a4b0-d87d726d834f could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:19.597683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-06c8ead3-173d-45fc-8305-8bc5a113c3fc tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f" status: 404 len: 111 microversion: 2.1 time: 0.043830 Sep 19 16:49:19.597683 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 919/2058] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:19 2022] GET /compute/v2.1/servers/4afae149-ed22-4a06-a4b0-d87d726d834f => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:19.597946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:19.597946 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:19.604718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-407b545e-742c-4a1d-bd2c-7df885ed5649 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] Calling method '.version_select at 0x7fadc30ea940>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:19.626677 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-407b545e-742c-4a1d-bd2c-7df885ed5649 tempest-TestMinimumBasicScenario-220534008 tempest-TestMinimumBasicScenario-220534008-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-2051475603" status: 202 len: 0 microversion: 2.1 time: 0.028226 Sep 19 16:49:19.627359 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 920/2059] 10.222.0.57 () {58 vars in 1272 bytes} [Mon Sep 19 16:49:19 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-2051475603 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:49:19.747310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:19.758577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-666d0e54-9544-4cee-ada0-88a731437528 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.189834 Sep 19 16:49:19.759624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1140/2060] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:19 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:20.080237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:20.081478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:20.089550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.090258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.090986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:20.091926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.092526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.092968 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:20.176310 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:20.186117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5043a45e-98b2-4fe9-9dea-23b776fd5f2b tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1271 microversion: 2.1 time: 0.112331 Sep 19 16:49:20.190455 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 921/2061] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:20 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1271 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:49:20.465154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:20.465938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:20.507452 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.510183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.510833 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:20.515068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.515537 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.515967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:20.733758 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:20.749016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:20.751447 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:20.752931 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6941df22-4e99-451f-b8c2-6d0a5dffca8f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.294372 Sep 19 16:49:20.753735 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1141/2062] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:20 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 296 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:20.788081 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:20.788713 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:20.798999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.800365 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.803829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:20.803829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:20.803829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:20.803829 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:21.213560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:21.214883 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:21.232007 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:21.239528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:21.241398 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:21.242769 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:21.243953 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:21.244989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:21.245747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:21.249992 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-65c46898-74b9-477d-957c-e9e6212c58b3 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1522 microversion: 2.1 time: 0.466635 Sep 19 16:49:21.250604 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 922/2063] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:20 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1522 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:21.297873 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=96,nova_cell1:UPDATE=2,nova_cell1:INSERT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:21.411002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:21.426268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cdadcbf1-42fa-4311-b29a-c86e6435c704 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1277 microversion: 2.1 time: 0.217286 Sep 19 16:49:21.427535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1142/2064] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:21 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1277 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:21.504295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c5d17c4e-bcff-42fa-95c7-f7df3fb97a42", "name": "network-vif-deleted", "tag": "a3ce130c-3698-4fbe-a5ac-58db4bc770fe"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:21.517232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:21.517838 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:21.518115 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:21.550949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] Creating event network-vif-deleted:a3ce130c-3698-4fbe-a5ac-58db4bc770fe for instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 on hv-cinder-85837 Sep 19 16:49:21.572014 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-41323efc-f61a-467e-9793-e39106d98186 req-f94c405f-9553-4d26-93c0-c40356e7778e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.068187 Sep 19 16:49:21.574581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 923/2065] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 74 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:21.793235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:21.795362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:21.823948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:21.824547 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:21.824998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:21.828087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:21.828582 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:21.830416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.265680 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:22.279847 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:22.280522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:22.282087 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:22.289588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:22.294922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-581dfbd0-b7a3-434a-86fd-6c917f097898 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.509064 Sep 19 16:49:22.296002 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.297509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1143/2066] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:21 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:22.302542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.302998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.303745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.304201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.305155 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.425210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.426043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.426904 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.428553 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lazy-loading 'fault' on Instance uuid c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:22.451585 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:22.452472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:22.467886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.468684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.469325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.470289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.470939 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.471524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.596609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8cd9c727c0b444cb83497ea4f78e24d0', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '0e35510b626e46aaae189f31b27ed663', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:22.612414 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-df16593b-37de-4a9c-a46b-070bd6298743 tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 200 len: 1311 microversion: 2.1 time: 0.338725 Sep 19 16:49:22.613008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 924/2067] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:22 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 1311 bytes in 340 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:22.904492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:22.905190 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "name": "network-vif-plugged", "status": "completed", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:22.918793 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2669a703-bc6f-4c33-a300-ca27e279db53 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1319 microversion: 2.1 time: 0.471024 Sep 19 16:49:22.919599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1144/2068] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:22 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1319 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:49:22.930652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:22.930652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:22.930652 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:22.965131 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] Creating event network-vif-plugged:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:49:22.986366 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-29381ec4-2ce0-4614-b3c8-9730e92f71a7 req-d4816554-0676-46ea-b4d9-7c39c3bfa9d2 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.353923 Sep 19 16:49:22.989056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 925/2069] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:23.325269 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:23.327727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:23.358902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.360404 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.362079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.363115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.364212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.364815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.608685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:23.640235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:23.641685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:23.648364 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:23.650770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] [instance: c5d17c4e-bcff-42fa-95c7-f7df3fb97a42] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:23.654644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c08a8575-f84e-4485-af21-1a0942e6b4db tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.333682 Sep 19 16:49:23.655904 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1145/2070] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:23 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:23.696722 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.697335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.701503 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.704469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.705250 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.705892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.733620 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] HTTP exception thrown: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Request method failure captured: Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 HTTP/1.1 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU4XZ-dembB8LJNZlvm6l5emS6V52pb2ARNav-LkFyYuRPrEyZ8QGlNYnB5U4Ly6xieQKgxt535USuInbYRdoCf0MO-ecduneINcBgiwnEcLKHlxV80mg9Flk7WCo4m28tvD7A99l93P7wqS0i2L7TcfNNWI5O_aRtzE-is_RYxqi4 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:49:23.740320 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/0e35510b626e46aaae189f31b27ed663"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_0e35510b626e46aaae189f31b27ed663"}]}] Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 0e35510b626e46aaae189f31b27ed663 Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-304726397 Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 8cd9c727c0b444cb83497ea4f78e24d0 Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-ServerRescueNegativeTestJSON-304726397-project Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. Sep 19 16:49:23.742884 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'c5d17c4e-bcff-42fa-95c7-f7df3fb97a42'} Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:49:23.743794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. Sep 19 16:49:23.745494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:49:23.748606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] Returning 404 to user: Instance c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:49:23.749410 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-bde7eb20-1d75-4252-a0c0-047b0bc82eac tempest-ServerRescueNegativeTestJSON-304726397 tempest-ServerRescueNegativeTestJSON-304726397-project] 10.222.0.57 "GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42" status: 404 len: 111 microversion: 2.1 time: 0.110328 Sep 19 16:49:23.755287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 926/2071] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:23 2022] GET /compute/v2.1/servers/c5d17c4e-bcff-42fa-95c7-f7df3fb97a42 => generated 111 bytes in 117 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:49:23.911527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d", "tag": "92bc1e86-1581-4928-8cfe-ae5f7719a8f6"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:23.934633 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.937579 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.945394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:23.946292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:23.957080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.021s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.961844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.963515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.963975 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.965900 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:23.966401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:23.967689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:23.988871 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] Creating event network-changed:92bc1e86-1581-4928-8cfe-ae5f7719a8f6 for instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d on hv-cinder-85837 Sep 19 16:49:24.007036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-700e04cd-5b86-4baa-b181-d860287f5951 req-e4cbad56-a978-4412-8f94-565f9586659c service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.098974 Sep 19 16:49:24.010592 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1146/2072] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:24.323200 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:24.338330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3585fb6b-3c27-4698-b125-4e7920040ed1 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1319 microversion: 2.1 time: 0.397650 Sep 19 16:49:24.349646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 927/2073] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:23 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1319 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:49:24.690128 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:24.693770 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:24.714692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:24.716508 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:24.717192 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:24.719217 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:24.719983 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:24.721442 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:25.202348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:25.212232 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:25.218820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:25.220426 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-619f46c1-d215-425d-bb66-c660bc721fa8 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.540234 Sep 19 16:49:25.239122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1147/2074] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:24 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 546 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:25.379809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:25.381068 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:25.392895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:25.403894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:25.404532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:25.405444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:25.406065 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:25.406814 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:26.258590 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:26.259218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:26.275164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:26.275958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:26.278768 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:26.280147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:26.280935 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:26.281465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:26.768938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:26.788847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:26.797581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:26.799058 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-729447a0-c4ae-409f-ab11-407d00e00198 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.550030 Sep 19 16:49:26.805737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:26.809432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:26.813131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1148/2075] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:26 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:26.817905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-6a68b249-4d2f-477e-aba3-75e3d8e0183d tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1319 microversion: 2.1 time: 1.449556 Sep 19 16:49:26.828517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 928/2076] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:25 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1319 bytes in 1461 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:27.833599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:27.834479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:27.848229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:27.858136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:27.859481 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:27.862479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=88,nova_api:DELETE=2,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:27.863527 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:27.868088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:27.871834 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:27.874651 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:27.875207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:27.885874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:27.886539 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:27.887067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:27.890678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:27.892606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:27.893142 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:28.291028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:28.317089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:28.318683 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:28.319924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-1b83bdce-0393-4920-a2bb-dd33c6845f42 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.490730 Sep 19 16:49:28.320490 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1149/2077] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:27 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:28.659656 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:28.669705 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-95bb134e-9b61-4208-8b7f-ace4edbcbc1f tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1319 microversion: 2.1 time: 0.819748 Sep 19 16:49:28.670301 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 929/2078] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:27 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1319 bytes in 821 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:29.210158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d", "name": "network-vif-plugged", "status": "completed", "tag": "92bc1e86-1581-4928-8cfe-ae5f7719a8f6"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:29.234599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:29.236646 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:29.237202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:29.238672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:29.270624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] Creating event network-vif-plugged:92bc1e86-1581-4928-8cfe-ae5f7719a8f6 for instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d on hv-cinder-85837 Sep 19 16:49:29.287914 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-f96f1a5f-aaa4-487d-bcd7-c113e42e3afc req-7b233399-afe4-4a56-9113-20e6464bc901 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.081256 Sep 19 16:49:29.289499 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1150/2079] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:29 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:29.352572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:29.354578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:29.365892 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:29.366945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:29.373317 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:29.374551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:29.376527 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:29.379135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:29.380816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:29.395614 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:29.700687 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:29.701464 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:29.722073 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:29.725723 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:29.726551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:29.729429 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:29.730364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:29.732126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:29.921923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:29.938770 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:29.943956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:29.945728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-7046f65e-3a32-448c-bb79-e684aa36f741 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.596742 Sep 19 16:49:29.956982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 930/2080] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:29 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 609 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:30.122255 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:30.139434 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-69b715a6-832f-4ab2-a8a7-d384c3927525 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1319 microversion: 2.1 time: 0.443581 Sep 19 16:49:30.139895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1151/2081] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:29 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1319 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:49:30.987606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:30.989709 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:31.009595 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.010346 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.010554 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.011217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.011750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.012122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.169074 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:31.170643 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:31.186119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.187107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.187752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.188749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.189456 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.189888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.408581 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:31.418664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ecd9cb09-59ff-4a56-a528-79a2d979de8e tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1508 microversion: 2.1 time: 0.257161 Sep 19 16:49:31.420000 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1152/2082] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:49:31 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1508 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:49:31.443413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a3896f94-6560-4703-b108-0df62388991c", "device": "/dev/sdb"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:31.446986 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:31.455695 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.456494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.457092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.457958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:31.458639 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:31.459231 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:31.669372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:31.695121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:31.698222 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:31.701733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c88d0182-d5ea-4324-8c3c-3f69c4cb03e4 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.717806 Sep 19 16:49:31.703389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 931/2083] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:30 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:31.910594 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-65688daf-51ca-4792-9823-223b346788ef tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "POST /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.469756 Sep 19 16:49:31.920656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1153/2084] 10.222.0.57 () {60 vars in 1323 bytes} [Mon Sep 19 16:49:31 2022] POST /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d/os-volume_attachments => generated 194 bytes in 474 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:49:32.732483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:32.733767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:32.744352 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:32.748750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:32.749103 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:32.753845 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:32.755789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:32.759735 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:33.023544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:33.034868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:33.037759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:33.043205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-c25f0134-2292-40e2-94d2-c822cecd3a73 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.315002 Sep 19 16:49:33.061388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 932/2085] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:32 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:34.073263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:34.074652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:34.095354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:34.096205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:34.096881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:34.099749 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:34.101063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:34.102912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:34.465718 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:34.479149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:34.488375 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:34.494888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b8432d12-a8a3-4928-9694-2601a78bf6f9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.426560 Sep 19 16:49:34.497492 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1154/2086] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:34 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 430 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:35.522660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:35.523483 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:35.538624 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:35.542846 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:35.546082 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:35.548313 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:35.549104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:35.550167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:35.841560 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:35.857906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:35.860117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:35.861803 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-de4590eb-3d08-485c-a365-2840c96b7334 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.343141 Sep 19 16:49:35.878175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 933/2087] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:35 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:36.912811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:36.917809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:36.937654 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=95,nova_api:UPDATE=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:36.941275 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:36.943596 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:36.944347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:36.946419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:36.947850 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:36.948942 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:37.154086 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:37.169518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:37.169518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:37.171360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-e9bb6cb6-8902-46c2-a44f-636cbaa8af63 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.279723 Sep 19 16:49:37.190184 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1155/2088] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:36 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:38.200456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:38.202295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:38.222728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:38.224858 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:38.225394 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:38.226946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:38.227506 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:38.227948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:38.402349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:38.412523 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:38.414143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:38.415469 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-959d5fac-3f8c-4357-a9cc-a0acbf110835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.222767 Sep 19 16:49:38.416119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 934/2089] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:38 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:39.441182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:39.442460 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:39.450080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:39.450601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:39.451249 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:39.451966 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:39.452418 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:39.452841 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:39.632541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:39.666806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:39.670104 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:39.671816 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f34239d6-ed59-4f4d-ac84-233fa0eaced9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1740 microversion: 2.20 time: 0.234570 Sep 19 16:49:39.673609 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1156/2090] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:39 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1740 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:40.703326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:40.704442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:40.715262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:40.715844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:40.716395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:40.717116 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:40.717575 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:40.718011 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:40.861771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:40.870980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:40.872431 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:40.873615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2d8b426-93ee-41b5-9fed-436451fc8314 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1722 microversion: 2.20 time: 0.177197 Sep 19 16:49:40.874177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 935/2091] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:40 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1722 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:41.920974 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:41.921767 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:41.929427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:41.930063 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:41.930629 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:41.931672 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:41.932136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:41.932564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:42.104660 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:42.113013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:42.114588 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:42.115745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9dcdc821-4688-4b9c-ab87-d6e8ce3decbc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1722 microversion: 2.20 time: 0.202495 Sep 19 16:49:42.116327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1157/2092] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:41 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1722 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:43.144027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:43.144643 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:43.151332 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:43.151811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:43.152246 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:43.152940 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:43.153391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:43.153823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:43.374206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:43.383248 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:43.384702 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:43.385878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-214f6685-9a84-42b5-aa81-6708033241d0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1722 microversion: 2.20 time: 0.245803 Sep 19 16:49:43.386647 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 936/2093] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:43 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1722 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:44.411092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:44.411818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:44.425922 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:44.428364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:44.431957 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:44.432868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:44.433846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:44.435262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:44.588687 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "name": "network-vif-unplugged", "status": "completed", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:44.621782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:44.623216 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:44.624210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:44.637763 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] Creating event network-vif-unplugged:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:49:44.646139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-272ee5cc-94bf-4f44-9af2-d8dc25a60805 req-ed4223f7-c0e0-435c-b4bb-9e96d19562f7 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.063141 Sep 19 16:49:44.648205 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 937/2094] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:49:44 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:49:44.723666 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:44.738325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:44.740624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:44.742147 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-89cf6844-fc5d-4f2b-8040-e3b5b87c13fc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1725 microversion: 2.20 time: 0.335183 Sep 19 16:49:44.743360 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1158/2095] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:44 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1725 bytes in 337 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:44.773137 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9aba1fa5-0f94-49a3-b427-de32d3354a4c"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:44.776294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:44.784151 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:44.785252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:44.786183 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:44.787376 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:44.788123 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:44.789243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:45.105544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-63d2ec51-e869-443f-8ae4-16d1fefe05c1 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.341024 Sep 19 16:49:45.106198 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 938/2096] 10.222.0.57 () {62 vars in 1364 bytes} [Mon Sep 19 16:49:44 2022] POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments => generated 188 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 19 16:49:45.239131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:45.239846 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:45.246467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:45.246948 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:45.247393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:45.248080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:45.248524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:45.248943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:45.251933 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:46.781505 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-570ec778-5fb7-4896-b06e-22cdd1aadf08 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments/9aba1fa5-0f94-49a3-b427-de32d3354a4c" status: 202 len: 0 microversion: 2.20 time: 1.547477 Sep 19 16:49:46.782226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1159/2097] 10.222.0.57 () {60 vars in 1457 bytes} [Mon Sep 19 16:49:45 2022] DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments/9aba1fa5-0f94-49a3-b427-de32d3354a4c => generated 0 bytes in 1550 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:49:46.846914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:49:46.847522 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:46.855397 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:46.855887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:46.856350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:46.857044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:46.857494 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:46.857923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:46.902804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120663) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 19 16:49:46.973932 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8ba4220d-81c1-491c-8e24-6926253f0d31 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/action" status: 202 len: 0 microversion: 2.20 time: 0.131009 Sep 19 16:49:46.974758 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 939/2098] 10.222.0.57 () {62 vars in 1319 bytes} [Mon Sep 19 16:49:46 2022] POST /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:49:46.983831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:46.984187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:46.990746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:46.991277 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:46.991713 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:46.992423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:46.992877 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:46.993297 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:47.144340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:47.153612 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:47.155119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:47.156282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-60225505-80d3-4d9e-97fb-5842cfc0a7e0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1673 microversion: 2.20 time: 0.176902 Sep 19 16:49:47.156835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1160/2099] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:46 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1673 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:48.181885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:48.182529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:48.188467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:48.188953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:48.189386 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:48.190074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:48.190566 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:48.190990 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:48.322877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:48.331030 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:48.332478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:48.333630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a7a0609-a75c-43ce-b01b-b07402d30a38 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1675 microversion: 2.20 time: 0.156492 Sep 19 16:49:48.334197 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 940/2100] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:48 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1675 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:49.363002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:49.363551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:49.370162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:49.370686 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:49.371116 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:49.371815 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:49.372533 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:49.373324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:49.506130 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:49.514943 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:49.516430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:49.517589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-27f0c5ef-7d95-4415-a192-22aab8bf5029 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1675 microversion: 2.20 time: 0.158600 Sep 19 16:49:49.518148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1161/2101] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:49 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1675 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:50.544917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:50.545533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:50.552162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:50.552640 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:50.553083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:50.553789 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:50.554239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:50.554923 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:50.715964 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:50.725787 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:50.727950 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:50.729161 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dc5d5bb2-f49a-4617-88f7-00ce5cac8d7f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1675 microversion: 2.20 time: 0.188306 Sep 19 16:49:50.730406 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 941/2102] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:50 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1675 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:49:51.770294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:51.771146 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:51.788346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:51.788938 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:51.789555 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:51.790187 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:51.790678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:51.791290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:52.004622 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:52.015494 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:52.017559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:52.018825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3c0c60d6-256c-48ee-9e76-e88b1ffd97c6 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.262387 Sep 19 16:49:52.019416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1162/2103] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:51 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 264 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:53.048612 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:53.049302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:53.060046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:53.061083 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:53.061889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:53.062926 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:53.063893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:53.064570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:53.090588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:53.343799 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:53.355943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:53.357596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:53.359117 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d607fef1-5042-456b-b58b-c2d4d761cdc2 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.316162 Sep 19 16:49:53.360367 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 942/2104] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:53 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 319 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:54.393408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:54.396590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:54.397257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:54.399812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:54.414914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:54.414914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:54.414914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:54.414914 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:54.420164 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:54.420697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:54.421875 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:54.422985 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:54.429389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:54.431123 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:54.431915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:54.432278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:54.609008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lazy-loading 'services' on Instance uuid 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:49:54.729342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:54.744719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:54.749292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:54.750640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7f18a088-8512-47cd-8bb5-3085aeb71db9 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.363364 Sep 19 16:49:54.751207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1163/2105] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:54 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:54.822412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-df9768d2-9b26-463d-a5b5-bd4825695c39 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d/os-volume_attachments/a3896f94-6560-4703-b108-0df62388991c" status: 202 len: 0 microversion: 2.1 time: 0.433314 Sep 19 16:49:54.822412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 943/2106] 10.222.0.57 () {58 vars in 1416 bytes} [Mon Sep 19 16:49:54 2022] DELETE /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d/os-volume_attachments/a3896f94-6560-4703-b108-0df62388991c => generated 0 bytes in 436 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:49:55.776120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:55.777868 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:55.785039 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:55.785746 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:55.786274 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:55.787226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:55.787852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:55.788332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:56.003578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:56.016338 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:56.019263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:56.036736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ee7a55bc-fae8-4194-9b7b-a17feed7edfc tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.254642 Sep 19 16:49:56.036736 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1164/2107] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:55 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:57.096294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:57.098733 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:57.122666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:57.123564 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:57.126707 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:57.127720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:57.128692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:57.129330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:57.399883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:57.407053 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:57.408720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:57.410056 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d3285d0b-99bc-4f84-8f63-05c9844b490b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.327295 Sep 19 16:49:57.430470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 944/2108] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:57 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:58.443919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:58.444517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:58.455406 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:58.455870 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:58.456302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:58.456995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:58.457438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:58.457856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:58.649821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:58.664952 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:58.669372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:58.685796 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4291a92b-2c88-4d62-b19f-60b2a6342011 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.247112 Sep 19 16:49:58.694934 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1165/2109] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:58 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:49:59.731022 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:49:59.731823 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:49:59.734399 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:49:59.740973 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:59.741798 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:59.743019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:59.743942 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:49:59.744594 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:49:59.745378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:49:59.935751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:59.945260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:59.947043 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:49:59.948441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d79682fa-7f83-40da-9c35-7893bd1a54ec tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.221346 Sep 19 16:49:59.948994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 945/2110] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:49:59 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:00.973804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:00.975156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:00.982806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:00.983669 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:00.984101 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:00.985050 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:00.985552 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:00.986198 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:01.142070 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:01.151372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:01.152824 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:01.153982 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-eeb15fef-111a-44f2-a5c3-7f4b16aa532c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.184620 Sep 19 16:50:01.154564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1166/2111] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:00 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:02.182027 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:02.182673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:02.189258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:02.189743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:02.190189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:02.190911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:02.191363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:02.191790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:02.339236 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:02.347617 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:02.349167 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:02.350484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-97df81e4-7d19-4501-b483-271931b09f21 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.173540 Sep 19 16:50:02.351128 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 946/2112] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:02 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:03.377453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:03.378396 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:03.385291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.385865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.386364 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.387226 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.387744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.388358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.538475 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:03.548790 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:03.550353 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:03.551802 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-91351a76-b65c-4335-b1e7-86f6c5b7984f tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.178351 Sep 19 16:50:03.552371 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1167/2113] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:03 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:03.595864 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:03.596467 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:03.603314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.603804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.604252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.604949 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.605401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.605883 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.632867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:50:03.725395 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-92eb3928-acb1-40e0-9ab1-1c85013764cc tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "DELETE /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 204 len: 0 microversion: 2.1 time: 0.132848 Sep 19 16:50:03.725997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 947/2114] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:50:03 2022] DELETE /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:50:03.735342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:03.752312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:03.765162 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.765571 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.766169 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.767124 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:03.767694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:03.768273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:03.909118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:03.918250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-acf22265-d834-4ebe-9000-b27050ee23f9 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1514 microversion: 2.1 time: 0.187277 Sep 19 16:50:03.919165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1168/2115] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:03 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1514 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:04.579750 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:04.580446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:04.587046 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:04.587533 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:04.587967 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:04.588657 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:04.589104 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:04.589529 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:04.728036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:04.737286 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:04.739015 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:04.740363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0323d988-316d-47a2-ab1d-b46b6805cf0e tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.164963 Sep 19 16:50:04.741192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 948/2116] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:04 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:04.944156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:04.944776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:04.952693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:04.953243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:04.953710 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:04.954463 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:04.955005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:04.955574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:05.089599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:05.098212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-debea047-146f-49e8-a3cf-020c9f6509a6 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1514 microversion: 2.1 time: 0.157859 Sep 19 16:50:05.099185 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1169/2117] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:04 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1514 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:05.784955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:05.787774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:05.795626 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:05.796108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:05.796711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:05.797650 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:05.798266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:05.798960 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:06.103121 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:06.113242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:06.115169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:06.116700 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-60ca04ca-e2dc-40bc-8df8-867624cae65b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.346656 Sep 19 16:50:06.122420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:06.123094 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:06.123593 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 949/2118] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:05 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:06.131373 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:06.132207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:06.133529 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:06.134488 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:06.135111 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:06.135940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:06.291628 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'e1c0b2cfaa9f485f9116e012c62120fb', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5fc6461a081c479fa20c9fb15a2b1575', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:06.300472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-43cd93b4-94c9-458c-ba23-9c5a1548d39c tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 200 len: 1514 microversion: 2.1 time: 0.182019 Sep 19 16:50:06.301561 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1170/2119] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:06 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 1514 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:50:06.492818 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d", "name": "network-vif-deleted", "tag": "92bc1e86-1581-4928-8cfe-ae5f7719a8f6"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:06.515630 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:06.516108 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:06.516570 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:06.531631 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] Creating event network-vif-deleted:92bc1e86-1581-4928-8cfe-ae5f7719a8f6 for instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d on hv-cinder-85837 Sep 19 16:50:06.536876 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-ea564cc3-cd77-42c8-90f6-1d981d06bd31 req-7ad95959-7773-4055-ab1e-fdb46ff15970 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049114 Sep 19 16:50:06.537565 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 950/2120] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:50:06.875616 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "name": "network-vif-plugged", "status": "completed", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:06.892149 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:06.893310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:06.894144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:06.920296 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] Creating event network-vif-plugged:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:50:06.931026 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-4911cffe-1955-4b9c-9c89-764c1a027753 req-6f7c8cdf-6dba-4444-ad74-0a7368f2d14a service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059100 Sep 19 16:50:06.932339 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1171/2121] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:50:07.144692 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:07.145697 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:07.153870 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:07.154742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:07.155668 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:07.156809 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:07.157444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:07.158169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:07.327805 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:07.328652 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] [instance: 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:07.331841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:07.337753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=34 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:07.340526 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:07.341999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:07.343211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2538069f-9dc3-4166-9e55-3a99f03d399b tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.203950 Sep 19 16:50:07.343765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 951/2122] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:07 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:07.344341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:07.344840 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:07.345300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:07.345970 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:07.346636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:07.347453 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:07.362978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] HTTP exception thrown: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Request method failure captured: Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d HTTP/1.1 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJYEWDTDTbrQcTJW62sbXG9JYj6prG771amK6BDHe050MiON0Zvs7CYcVhwwJQ5H_6g3afVJuZrViHPE3pP-D5Og0tzKPkbh4PtBnW_ElNCBmvEOJVJsBvYX4HZdcUvX2Omo7vtGfHMXyz9Jqfajr9-CYJHum7eS4Ra-TZNbMryKn90 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:50:07.364664 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/5fc6461a081c479fa20c9fb15a2b1575"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_5fc6461a081c479fa20c9fb15a2b1575"}]}] Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 5fc6461a081c479fa20c9fb15a2b1575 Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1048443447 Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: e1c0b2cfaa9f485f9116e012c62120fb Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesSnapshotTestJSON-1048443447-project Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. Sep 19 16:50:07.366515 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d'} Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:50:07.368095 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. Sep 19 16:50:07.369818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:07.371382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] Returning 404 to user: Instance 39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:50:07.371382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bd94df08-63fe-45b6-8aa8-dd1fd4200659 tempest-VolumesSnapshotTestJSON-1048443447 tempest-VolumesSnapshotTestJSON-1048443447-project] 10.222.0.57 "GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d" status: 404 len: 111 microversion: 2.1 time: 0.043334 Sep 19 16:50:07.371382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1172/2123] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:07 2022] GET /compute/v2.1/servers/39c18cd9-fe8e-4a55-9a0d-33e1673c8f9d => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:50:08.370844 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:08.371477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:08.377894 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:08.378385 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:08.378832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:08.379633 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:08.379980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:08.380402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:08.516679 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:08.525877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:08.527484 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:08.528669 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-44be8530-22ec-407c-8dd7-b142c1f55727 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.162429 Sep 19 16:50:08.529242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 952/2124] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:08 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:09.555389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:09.556137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:09.563313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:09.564082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:09.564524 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:09.565262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:09.565701 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:09.566197 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:09.697908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:09.706075 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:09.707575 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:09.708721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c5328b10-3604-4832-afcc-b2fa9aea99fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.157973 Sep 19 16:50:09.709266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1173/2125] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:09 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:10.734464 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:10.735073 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:10.741378 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:10.742337 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:10.742797 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:10.743476 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:10.743915 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:10.744330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:10.880302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:10.889302 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:10.890827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:10.891997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-5e544950-4973-47f9-8349-54d4b091b835 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.161504 Sep 19 16:50:10.892544 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 953/2126] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:10 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:11.917797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:11.918385 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:11.924835 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:11.925312 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:11.925747 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:11.926465 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:11.926918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:11.927346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:11.946110 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=2,nova_cell1:UPDATE=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:12.065102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:12.074112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:12.075679 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:12.076842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-0bb44a86-dc36-4006-9efe-40319328562d tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1731 microversion: 2.20 time: 0.163646 Sep 19 16:50:12.077403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1174/2127] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:11 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1731 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:13.104075 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:13.105330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:13.115868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:13.117016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:13.117761 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:13.119000 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:13.119822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:13.120588 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:13.253851 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:13.264440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:13.266592 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:13.268238 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-591b6318-0528-448e-b2bd-13c8b3b8ea57 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1718 microversion: 2.20 time: 0.169609 Sep 19 16:50:13.269295 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 954/2128] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:13 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1718 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:21.084467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-116597324", "imageRef": "a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e", "flavorRef": "452"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:21.139278 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.network.neutron [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] validate_networks() for None {{(pid=120664) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:50:21.458176 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Flavor limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:50:21.458642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Image limits 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:50:21.459114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Flavor pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:50:21.459522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Image pref 0:0:0 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:50:21.460021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.virt.hardware [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120664) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:50:21.469649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:21.470135 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:21.470620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:21.471340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:21.471792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:21.472443 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:21.489220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Getting quotas for project 9026e39904fa45eb9984dc3d687e7a08. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:50:21.493238 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Getting quotas for user c4bfbedbe6cd494984e181a29c36ed8c and project 9026e39904fa45eb9984dc3d687e7a08. Resources: {'ram', 'cores', 'instances'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:50:21.504035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Going to run 1 instances... {{(pid=120664) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:50:21.506344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='a4a9b84f-0a15-4153-9fd0-d5dcfd0c738e',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120664) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:50:21.553856 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-be1ed195-6c14-4585-bb67-cba4679be3bb tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.473218 Sep 19 16:50:21.554762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1175/2129] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:50:21 2022] POST /compute/v2.1/servers => generated 380 bytes in 475 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:50:21.566095 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4a32b1bd-bf6a-457c-8a8f-bef0f151e77c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:21.566918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4a32b1bd-bf6a-457c-8a8f-bef0f151e77c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:21.621555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a32b1bd-bf6a-457c-8a8f-bef0f151e77c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:21.628674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4a32b1bd-bf6a-457c-8a8f-bef0f151e77c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1214 microversion: 2.1 time: 0.066521 Sep 19 16:50:21.629189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 955/2130] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:21 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1214 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:22.652268 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:22.652884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:22.659678 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:22.660178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:22.660603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:22.661281 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:22.661727 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:22.662150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:22.728137 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:22.735745 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b6507e19-bd96-45d9-84f7-55cf8069eb03 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1218 microversion: 2.1 time: 0.087891 Sep 19 16:50:22.736310 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1176/2131] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:22 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1218 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:23.758917 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:23.759491 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:23.761946 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=82,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:23.767350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:23.767835 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:23.768283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:23.768986 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:23.769440 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:23.769868 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:23.836023 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:23.843696 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-e2e70bcf-a13c-49a7-91c2-534de8a5fe13 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1218 microversion: 2.1 time: 0.089398 Sep 19 16:50:23.844258 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 956/2132] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:23 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1218 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:24.866607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:24.867220 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:24.873640 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:24.874115 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:24.874574 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:24.875254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:24.875699 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:24.876134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:24.944218 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:24.952690 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-65f88f47-fded-47f2-bd81-8c0ee5770575 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1266 microversion: 2.1 time: 0.089992 Sep 19 16:50:24.953270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1177/2133] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:24 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1266 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:25.976441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:25.977596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:25.993623 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:25.994177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:25.994688 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:25.995508 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:25.996107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:25.996646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:26.295448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:26.305284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b40e9f11-79b5-404d-be2c-21d4d06858dd tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.333577 Sep 19 16:50:26.306432 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 957/2134] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:25 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:26.823958 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "0ce19550-ec75-4ab4-9e32-133440a6ecb4", "tag": "1f491c8e-ed34-4c70-9049-5b972d73ef9a"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:26.834120 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:26.834516 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:26.834990 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:26.846262 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] Creating event network-changed:1f491c8e-ed34-4c70-9049-5b972d73ef9a for instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 on hv-cinder-85837 Sep 19 16:50:26.851469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-c9b7236d-1a6a-4b55-b018-e6502f367bbb req-6a0267e2-4d5e-4db2-8c8f-5f5678599e76 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.031128 Sep 19 16:50:26.852569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1178/2135] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:50:27.329552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:27.330187 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:27.337092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:27.337572 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:27.338004 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:27.338719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:27.339175 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:27.339598 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:27.465460 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:27.473008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a7ad8b83-9f0c-40c2-9b6a-75b31dba5cf3 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.147513 Sep 19 16:50:27.473578 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 958/2136] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:27 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 149 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:28.497500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:28.498193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:28.504731 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:28.505202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:28.505627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:28.506300 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:28.506776 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:28.507193 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:28.644994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:28.655112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-dc02c386-d8ad-4d47-b9ee-27c905cdd58c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.161613 Sep 19 16:50:28.655684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1179/2137] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:28 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:29.681548 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:29.682451 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:29.688924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:29.689396 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:29.689822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:29.690528 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:29.690980 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:29.691446 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:29.906067 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:29.913773 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8a08531d-4ede-41a7-aca7-055f7f265cbc tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.236876 Sep 19 16:50:29.914405 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 959/2138] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:29 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:30.937801 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:30.938437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:30.944332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:30.944797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:30.945219 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:30.946551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:30.947003 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:30.947419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:31.078122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:31.085730 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-20f65097-5b37-434a-afe8-eac3d42f0ddf tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.151987 Sep 19 16:50:31.086293 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1180/2139] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:30 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:50:32.110794 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:32.111478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:32.113752 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:32.115162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=37,nova_api:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:32.120971 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:32.121453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:32.121895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:32.122622 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:32.123076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:32.123502 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:32.256345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:32.264439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-10599e36-5a54-4837-91db-c84e5612f5e7 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.157842 Sep 19 16:50:32.264999 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 960/2140] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:32 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:33.287656 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:33.288270 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:33.290246 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:33.290984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=6 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:33.297018 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:33.297830 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:33.298261 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:33.298973 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:33.299421 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:33.299845 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:33.427430 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:33.436076 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bd6f688e-54dd-46e0-9718-281eebd82b5c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.152303 Sep 19 16:50:33.436663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1181/2141] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:33 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:34.459283 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:34.459956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:34.466419 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:34.466898 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:34.467331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:34.468017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:34.468468 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:34.468893 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:34.594314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:34.601938 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4914a859-e25a-46d8-a72a-d42b4498dd54 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.146561 Sep 19 16:50:34.603036 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 961/2142] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:34 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:35.606514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ce19550-ec75-4ab4-9e32-133440a6ecb4", "name": "network-vif-plugged", "status": "completed", "tag": "1f491c8e-ed34-4c70-9049-5b972d73ef9a"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:35.625420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:35.625900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:35.626332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:35.626660 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:35.627100 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:35.637282 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:35.639131 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] Creating event network-vif-plugged:1f491c8e-ed34-4c70-9049-5b972d73ef9a for instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 on hv-cinder-85837 Sep 19 16:50:35.640673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:35.641995 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:35.643756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:35.644797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-e4e65a13-f314-437a-b0b4-68e3404cea90 req-f07d4847-dfd2-4fce-bbdc-8f0ae4538718 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043311 Sep 19 16:50:35.645354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:35.645693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1182/2143] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:35 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:50:35.646210 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:35.801568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:35.812287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-85a04f13-6337-4085-9965-0f5b2588e14a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.190035 Sep 19 16:50:35.813287 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 962/2144] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:35 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:36.836172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:36.837504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:36.844350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:36.844829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:36.845271 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:36.845959 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:36.846437 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:36.846865 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:36.983350 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:36.991224 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-7c449b42-6029-4eae-920e-e10182320c9c tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1314 microversion: 2.1 time: 0.159262 Sep 19 16:50:36.991792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1183/2145] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:36 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1314 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:38.013765 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:38.014429 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:38.021192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.021674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.022110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.022857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.023314 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.023748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.167135 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:38.175788 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-95151c67-0bd8-4090-9f78-bb2b343ef825 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1463 microversion: 2.1 time: 0.165919 Sep 19 16:50:38.176549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 963/2146] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:38 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1463 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:38.225569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:38.226365 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:38.228478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=35,nova_api:INSERT=3 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:38.233869 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.234354 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.234811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.235500 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.235954 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.236381 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.263976 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Going to try to terminate instance {{(pid=120664) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:50:38.349783 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f73d1834-b1bd-4271-a72a-2d624b6509d8 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "DELETE /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 204 len: 0 microversion: 2.1 time: 0.128465 Sep 19 16:50:38.350541 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1184/2147] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:50:38 2022] DELETE /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:50:38.359748 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:38.360357 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:38.366298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.366821 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.367254 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.368611 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:38.369057 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:38.369475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:38.505591 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:38.514543 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-87509741-dc05-4a6b-ae3f-f4fbeb0e615a tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1469 microversion: 2.1 time: 0.159084 Sep 19 16:50:38.515122 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 964/2148] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:38 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1469 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:39.540035 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:39.541078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:39.548698 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:39.549316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:39.549671 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:39.550792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:39.551624 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:39.552082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:39.735599 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:39.745154 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-88e56101-eb54-43dd-89c8-2f666ee9c880 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1469 microversion: 2.1 time: 0.210768 Sep 19 16:50:39.745737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1185/2149] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:39 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1469 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:40.641120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "0ce19550-ec75-4ab4-9e32-133440a6ecb4", "name": "network-vif-deleted", "tag": "1f491c8e-ed34-4c70-9049-5b972d73ef9a"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:40.653129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:40.653619 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:40.654049 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:40.670925 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] Creating event network-vif-deleted:1f491c8e-ed34-4c70-9049-5b972d73ef9a for instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 on hv-cinder-85837 Sep 19 16:50:40.682745 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-dbdb6177-e00d-4719-8616-96af0199568a req-53ed4161-02ed-4710-bcd1-fc9017a01ac6 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.043487 Sep 19 16:50:40.689437 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 965/2150] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:40 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:50:40.770677 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:40.771413 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:40.777941 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:40.778471 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:40.778908 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:40.779926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:40.780370 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:40.780794 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:40.830291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:40.831044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:40.831514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:40.832030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lazy-loading 'fault' on Instance uuid 0ce19550-ec75-4ab4-9e32-133440a6ecb4 {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:50:40.921752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c4bfbedbe6cd494984e181a29c36ed8c', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '9026e39904fa45eb9984dc3d687e7a08', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:40.936289 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-b7276c0d-bd61-4070-86b4-4ab4cf824228 tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 200 len: 1304 microversion: 2.1 time: 0.170382 Sep 19 16:50:40.943282 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1186/2151] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:40 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 1304 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:41.963284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:41.964070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] [instance: 0ce19550-ec75-4ab4-9e32-133440a6ecb4] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:41.970240 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:41.970754 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:41.971189 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:41.971944 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:41.972325 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:41.972741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:41.989714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] HTTP exception thrown: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Request method failure captured: Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 HTTP/1.1 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ3JGrUK9-0O83CPS2x81XmlZIypL5DjhB0RVyD1At4FgzBaWtby-xMI2VRFuoKASwX1EHmyB7hAZIraKC0PREYWo8fAYcjKGhq5dC40QuSPcmah4vVEfBpM4zryz27ucPMSM7jwTTL_PfbCxzz9-HT87kGq2X6EpjFt2myICQEyjqQ Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 9026e39904fa45eb9984dc3d687e7a08 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesNegativeTest-524994927 Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:50:41.991556 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/9026e39904fa45eb9984dc3d687e7a08"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/9026e39904fa45eb9984dc3d687e7a08"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/9026e39904fa45eb9984dc3d687e7a08"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_9026e39904fa45eb9984dc3d687e7a08"}]}] Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesNegativeTest-524994927 Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 9026e39904fa45eb9984dc3d687e7a08 Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesNegativeTest-524994927 Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesNegativeTest-524994927-project Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: c4bfbedbe6cd494984e181a29c36ed8c Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesNegativeTest-524994927-project Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. Sep 19 16:50:41.993479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '0ce19550-ec75-4ab4-9e32-133440a6ecb4'} Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:50:41.994478 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. Sep 19 16:50:41.996024 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:50:41.997475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] Returning 404 to user: Instance 0ce19550-ec75-4ab4-9e32-133440a6ecb4 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:50:41.997475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8e95c595-4aa6-4e03-b4dd-727f86e48a5d tempest-VolumesNegativeTest-524994927 tempest-VolumesNegativeTest-524994927-project] 10.222.0.57 "GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4" status: 404 len: 111 microversion: 2.1 time: 0.035156 Sep 19 16:50:41.997475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 966/2152] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:50:41 2022] GET /compute/v2.1/servers/0ce19550-ec75-4ab4-9e32-133440a6ecb4 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:50:53.526329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:53.529290 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:53.535707 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=12,nova_api:UPDATE=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:53.536601 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=72,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:53.539829 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.540875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.541649 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:53.542762 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.543389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.544068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:53.580828 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-61c04b01-74af-433b-8b39-b210f75f44fd tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.058734 Sep 19 16:50:53.582029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1187/2153] 10.222.0.57 () {60 vars in 1343 bytes} [Mon Sep 19 16:50:53 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780/os-volume_attachments => generated 25 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:50:53.663280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:53.663922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:53.666338 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=13 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:53.667275 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=58 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:50:53.674201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.674863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.675401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:53.676211 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.676780 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.677581 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:53.708412 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:50:53.801372 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b6655dfd-49b4-478b-a908-85d950aeafd0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 204 len: 0 microversion: 2.20 time: 0.142835 Sep 19 16:50:53.802454 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 967/2154] 10.222.0.57 () {60 vars in 1280 bytes} [Mon Sep 19 16:50:53 2022] DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 19 16:50:53.812716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:53.813337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:53.819564 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.820361 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.820831 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:53.821725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:53.822355 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:53.823002 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:54.061744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:54.071569 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:54.073313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:54.074806 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c60d9ac0-6903-41d7-b065-c45c57c97f83 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.267679 Sep 19 16:50:54.086485 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1188/2155] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:53 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 19 16:50:55.101615 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:55.102418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:55.110691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:55.110691 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:55.110998 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:55.111596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:55.112107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:55.112475 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:55.277114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:55.294079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:55.295936 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:55.297136 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a2159300-706e-4e77-809a-5efa2e72afe0 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1724 microversion: 2.20 time: 0.199844 Sep 19 16:50:55.297703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 968/2156] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:55 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1724 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:56.333382 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:56.338092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:56.348674 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:56.349423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:56.350047 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:56.350807 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:56.354703 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:56.355190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:56.860577 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:56.881122 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:56.885119 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'bd190d8e252342ca9232f7a0c966c715', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '344bf851ff7d471fba47bde9b6f7e219', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:50:56.887315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-3a952fad-4744-4774-a6e7-14c8ed8ba3c7 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 200 len: 1656 microversion: 2.20 time: 0.563804 Sep 19 16:50:56.888157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1189/2157] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:56 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 1656 bytes in 568 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 19 16:50:57.094071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "90999c63-0653-4625-85c8-9ee565c5d780", "name": "network-vif-deleted", "tag": "701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:50:57.143309 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:57.145008 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:57.146363 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:57.172547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] Creating event network-vif-deleted:701b8cc5-fa2a-4f3e-ae14-ef7595bca7d9 for instance 90999c63-0653-4625-85c8-9ee565c5d780 on hv-cinder-85837 Sep 19 16:50:57.188698 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-7f44dca1-2e2c-4c0d-b343-a805169551aa req-f7ae3cbc-4060-4755-b99f-08eb828b7565 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.100667 Sep 19 16:50:57.190018 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 969/2158] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:50:57 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:50:57.918792 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:57.919327 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:50:57.938030 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:57.939693 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:57.947962 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:57.949212 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:50:57.949923 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:50:57.951332 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:50:58.001132 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] HTTP exception thrown: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Request method failure captured: Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 HTTP/1.1 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU81EqY99T609uuwZr9a0sNsoO1V_0zQrwGV9SChkBUTfz8HXeTTUB8w4IvocWqxDJwsnWnMLXY0Ct47KAVDZkhqs20bNAkHi0REaMP2d51-mMiBqEWxa2Sd96fcOxr0jiXyl6uHdarWeygRDHXrca5WbxkjSAyj3ClfovshzB9HSY Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Openstack-Nova-Api-Version: 2.20 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:50:58.010203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_344bf851ff7d471fba47bde9b6f7e219"}]}] Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: bd190d8e252342ca9232f7a0c966c715 Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:50:58.012372 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '90999c63-0653-4625-85c8-9ee565c5d780'} Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:50:58.013444 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:50:58.015333 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:50:58.018565 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Returning 404 to user: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:50:58.021144 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-fcbf0ef6-4794-4330-bdde-bc0503303a1a tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 404 len: 111 microversion: 2.20 time: 0.108244 Sep 19 16:50:58.032953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-765aadc9-ff55-4ec9-a0e5-12631eae8a25 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '.version_select at 0x7fadc30ea940>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:50:58.042518 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1190/2159] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:50:57 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 111 bytes in 127 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:50:58.076319 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-765aadc9-ff55-4ec9-a0e5-12631eae8a25 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1898280496" status: 204 len: 0 microversion: 2.20 time: 0.047460 Sep 19 16:50:58.076945 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 970/2160] 10.222.0.57 () {60 vars in 1262 bytes} [Mon Sep 19 16:50:58 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1898280496 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 19 16:51:00.936781 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:00.937394 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:00.944235 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:00.944788 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:00.945852 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:00.949478 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:00.949967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:00.950888 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:00.971854 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] HTTP exception thrown: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Request method failure captured: Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 HTTP/1.1 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU81EqY99T609uuwZr9a0sNsoO1V_0zQrwGV9SChkBUTfz8HXeTTUB8w4IvocWqxDJwsnWnMLXY0Ct47KAVDZkhqs20bNAkHi0REaMP2d51-mMiBqEWxa2Sd96fcOxr0jiXyl6uHdarWeygRDHXrca5WbxkjSAyj3ClfovshzB9HSY Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Openstack-Nova-Api-Version: 2.20 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:51:00.974266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_344bf851ff7d471fba47bde9b6f7e219"}]}] Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: bd190d8e252342ca9232f7a0c966c715 Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:00.976021 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '90999c63-0653-4625-85c8-9ee565c5d780'} Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:51:00.977535 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:00.980237 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:51:00.982895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Returning 404 to user: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:51:00.982895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6510478c-9a54-411f-947e-94c4368b2c90 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 404 len: 111 microversion: 2.20 time: 0.043547 Sep 19 16:51:00.982895 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1191/2161] 10.222.0.57 () {60 vars in 1280 bytes} [Mon Sep 19 16:51:00 2022] DELETE /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 111 bytes in 49 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:51:00.990349 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:00.991296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 90999c63-0653-4625-85c8-9ee565c5d780] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:00.998524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:00.999114 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:00.999632 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:01.000589 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:01.001130 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:01.001636 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:01.020782 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] HTTP exception thrown: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Request method failure captured: Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 HTTP/1.1 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU81EqY99T609uuwZr9a0sNsoO1V_0zQrwGV9SChkBUTfz8HXeTTUB8w4IvocWqxDJwsnWnMLXY0Ct47KAVDZkhqs20bNAkHi0REaMP2d51-mMiBqEWxa2Sd96fcOxr0jiXyl6uHdarWeygRDHXrca5WbxkjSAyj3ClfovshzB9HSY Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Openstack-Nova-Api-Version: 2.20 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:51:01.025670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_344bf851ff7d471fba47bde9b6f7e219"}]}] Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: bd190d8e252342ca9232f7a0c966c715 Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:01.029102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '90999c63-0653-4625-85c8-9ee565c5d780'} Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:51:01.030994 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. Sep 19 16:51:01.035448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:51:01.038424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Returning 404 to user: Instance 90999c63-0653-4625-85c8-9ee565c5d780 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:51:01.038424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2970802f-ee88-4d10-8475-23d2ad75e97c tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780" status: 404 len: 111 microversion: 2.20 time: 0.050426 Sep 19 16:51:01.038424 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 971/2162] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:51:00 2022] GET /compute/v2.1/servers/90999c63-0653-4625-85c8-9ee565c5d780 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:51:01.370578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-a8f85115-147a-43ec-9e67-ff73eb060a9e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Action: 'create', calling method: .version_select at 0x7fadbfa2bca0>, body: {"keypair": {"name": "tempest-TestShelveInstance-1671444004", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBK2V6VV4wj+1mCUxZj/4MfY+n/6j/V6EVxG4GgN/qsChHgczPohcsmfOxVSHF3+UAUE2CL+aTVUotMQ8PQ2espKSPkc+eS8U/ZvzIB2JFIyI2WqFbXDtBG/57h23S4aQHg=="}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:51:01.388236 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a8f85115-147a-43ec-9e67-ff73eb060a9e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Getting quotas for project b62d50ba5e3d451dab48a4b6cdc50399. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:51:01.392389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.quota [None req-a8f85115-147a-43ec-9e67-ff73eb060a9e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Getting quotas for user 8a3f7b1c624141feab14c67fa39732a1 and project b62d50ba5e3d451dab48a4b6cdc50399. Resources: {'key_pairs'} {{(pid=120664) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:51:01.452457 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-a8f85115-147a-43ec-9e67-ff73eb060a9e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "POST /compute/v2.1/os-keypairs" status: 200 len: 397 microversion: 2.1 time: 0.266637 Sep 19 16:51:01.453049 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1192/2163] 10.222.0.57 () {60 vars in 1159 bytes} [Mon Sep 19 16:51:01 2022] POST /compute/v2.1/os-keypairs => generated 397 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:51:23.186442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-285485582", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-1671444004", "security_groups": [{"name": "tempest-secgroup-smoke-280956216"}], "networks": [{"uuid": "e2c54551-e637-4413-990d-7e615809d787"}], "block_device_mapping_v2": [{"uuid": "a89bd8d7-6295-45e6-a93c-f08329ad2aa9", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:51:23.197956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:23.198832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=24,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:23.466430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.network.neutron [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] validate_networks() for [('e2c54551-e637-4413-990d-7e615809d787', None, None, None, None, None)] {{(pid=120663) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 19 16:51:23.850511 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Flavor limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 19 16:51:23.850948 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Image limits 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 19 16:51:23.851428 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Flavor pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 19 16:51:23.851842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Image pref 0:0:0 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 19 16:51:23.852259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.virt.hardware [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120663) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 19 16:51:23.862033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:23.862550 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:23.862996 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:23.863719 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:23.864174 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:23.864606 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:23.881857 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Getting quotas for project b62d50ba5e3d451dab48a4b6cdc50399. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 19 16:51:23.885911 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.quota [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Getting quotas for user 8a3f7b1c624141feab14c67fa39732a1 and project b62d50ba5e3d451dab48a4b6cdc50399. Resources: {'ram', 'cores', 'instances'} {{(pid=120663) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 19 16:51:23.943885 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Going to run 1 instances... {{(pid=120663) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 19 16:51:23.946020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='a89bd8d7-6295-45e6-a93c-f08329ad2aa9',volume_size=None,volume_type=None)] {{(pid=120663) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 19 16:51:24.079169 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4ca83f40-4cb3-4619-b780-3fc9a4213b05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "POST /compute/v2.1/servers" status: 202 len: 405 microversion: 2.1 time: 0.896782 Sep 19 16:51:24.079817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 972/2164] 10.222.0.57 () {60 vars in 1147 bytes} [Mon Sep 19 16:51:23 2022] POST /compute/v2.1/servers => generated 405 bytes in 898 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 19 16:51:24.088509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=22 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:24.089272 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=4 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:24.090403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=21,nova_api:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:24.094407 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58eb377b-da6f-4a93-b770-3d101ad06a41 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:24.095032 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-58eb377b-da6f-4a93-b770-3d101ad06a41 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:24.162619 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58eb377b-da6f-4a93-b770-3d101ad06a41 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:24.170225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58eb377b-da6f-4a93-b770-3d101ad06a41 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1088 microversion: 2.1 time: 0.082930 Sep 19 16:51:24.173423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1193/2165] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:24 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1088 bytes in 84 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:51:25.194149 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:25.195270 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:25.202330 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:25.203035 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:25.203583 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:25.204439 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:25.205168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:25.205837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:25.291032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:25.301159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-83bcfbc0-547d-4cb7-bdeb-b40727ad8cc1 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1194 microversion: 2.1 time: 0.111897 Sep 19 16:51:25.302088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 973/2166] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:25 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1194 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:26.325855 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:26.326514 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:26.333158 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:26.333644 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:26.334084 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:26.334812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:26.335266 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:26.335692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:26.474254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:26.483335 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-9eb8e47b-5b6d-46ce-b7ec-e2e8b192c025 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.161966 Sep 19 16:51:26.483918 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1194/2167] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:26 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:26.638882 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:51:26.652902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:26.653384 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:26.653819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:26.665477 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] Creating event network-changed:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:51:26.670977 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-3213aeeb-82f0-416f-8e2a-1021795b916f req-b4157137-be53-4823-806b-3f767cc33172 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036563 Sep 19 16:51:26.671580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 974/2168] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:51:26 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:51:27.507820 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:27.508584 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:27.515940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:27.516821 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:27.517420 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:27.518362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:27.518899 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:27.519449 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:27.686097 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:27.694709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-39c944ab-4505-49ef-846b-773cda6d42c2 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.191362 Sep 19 16:51:27.695305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1195/2169] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:27 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:28.721206 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:28.721953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:28.728071 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:28.728728 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:28.729168 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:28.729849 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:28.730296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:28.730755 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:28.861110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:28.868827 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ad74cf50-2fd7-4cfe-a254-1d37ff26ce54 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.152302 Sep 19 16:51:28.869392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 975/2170] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:28 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:29.890811 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:29.891484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:29.897534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:29.898316 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:29.898517 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:29.899207 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:29.899663 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:29.900088 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:30.040190 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:30.048560 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-796aea68-a265-4a59-8424-adbce4c1bfca tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.161538 Sep 19 16:51:30.049126 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1196/2171] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:29 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:31.072234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:31.072802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:31.079816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:31.080292 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:31.080730 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:31.081421 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:31.081871 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:31.082296 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:31.213675 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:31.221389 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b36dc049-8970-4b3a-b1b4-082ea92d0d8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.153195 Sep 19 16:51:31.221953 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 976/2172] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:31 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:32.244566 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:32.245183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:32.252225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:32.252709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:32.253148 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:32.253847 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:32.254329 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:32.254779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:32.391118 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:32.398915 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ace4690e-5f09-468f-89e9-c9decb1d7f3f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.158179 Sep 19 16:51:32.399496 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1197/2173] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:32 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:33.422863 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:33.423542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:33.430025 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:33.430541 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:33.430974 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:33.431695 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:33.432383 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:33.432804 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:33.566703 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:33.575262 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a6cbe247-23ec-409a-b525-74cdbef1f43f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.156311 Sep 19 16:51:33.576442 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 977/2174] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:33 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:51:34.598721 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:34.599341 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:34.601752 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell0:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:34.608773 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:34.609259 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:34.609696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:34.610417 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:34.610875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:34.611305 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:34.749688 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:34.757951 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4aea1b0a-3c42-468f-9f95-14d32ab75645 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.163172 Sep 19 16:51:34.758576 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1198/2175] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:34 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:35.781413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:35.782033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:35.784233 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:35.784833 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell0:SELECT=6 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:51:35.791340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:35.791956 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:35.792297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:35.793764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:35.794417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:35.794762 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:35.929230 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:35.939219 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ec8d8375-32c5-4bd9-8fe1-87668c9af7bb tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.161633 Sep 19 16:51:35.939802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 978/2176] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:35 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:36.962033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:36.962697 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:36.969509 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:36.969998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:36.970825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:36.971157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:36.971607 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:36.972033 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:37.111544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:37.119366 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-209df482-7414-476f-a481-5973da779a1a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.161212 Sep 19 16:51:37.120044 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1199/2177] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:36 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:37.872234 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "name": "network-vif-plugged", "status": "completed", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:51:37.882402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:37.883038 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:37.883408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:37.895227 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] Creating event network-vif-plugged:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:51:37.900495 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-650986c1-99a4-42bf-8aef-62d98704afcd req-8f1a7108-89fc-4fc0-b78d-b7aecc25685f service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.032349 Sep 19 16:51:37.901398 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 979/2178] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:51:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:51:38.143732 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:38.144357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:38.151173 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:38.151522 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:38.152482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:38.153202 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:38.153653 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:38.154078 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:38.291760 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:38.299476 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-38bd2d8c-4841-4489-be2c-9a5d2b99d354 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1259 microversion: 2.1 time: 0.159736 Sep 19 16:51:38.300045 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1200/2179] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:38 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1259 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:39.323741 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:39.324435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:39.330517 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:39.330997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:39.331427 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:39.332102 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:39.332549 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:39.332966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:39.469802 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:39.477736 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4a0950e9-b0fa-41d1-b38d-78a6598e3ada tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1443 microversion: 2.1 time: 0.158736 Sep 19 16:51:39.478513 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 980/2180] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:39 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1443 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:39.499817 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:51:39.500427 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:51:39.507082 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:39.507559 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:39.507989 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:39.508667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:39.509114 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:39.509534 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:39.645467 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:51:39.653706 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f1808149-bd4e-448c-907c-cc0dae2e1ef3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1443 microversion: 2.1 time: 0.157681 Sep 19 16:51:39.654273 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1201/2181] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:51:39 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1443 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:51:41.989714 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:51:42.000150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:51:42.000551 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:51:42.000981 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:51:42.015905 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.compute.server_external_events [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] Creating event network-changed:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:51:42.021209 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [req-9204496a-75b7-4322-9a45-cf1fa8f8d119 req-f6688fe1-5948-4fb9-9719-decf5ddafe8e service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.201189 Sep 19 16:51:42.021834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 981/2182] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:51:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:53:37.141984 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=20 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:37.142667 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=48 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:37.143750 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:53:37.143900 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:53:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action (ip 10.222.0.57) !!! Sep 19 16:53:37.146028 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:53:37.146156 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:53:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action (ip 10.222.0.57) !!! Sep 19 16:53:37.152578 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:53:37.153107 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:37.166625 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:37.167425 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:37.168134 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:37.169340 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:37.170150 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:37.171448 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:37.195229 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.objects.instance [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lazy-loading 'flavor' on Instance uuid 4064c43e-48d4-42c9-a243-390cf8f8415b {{(pid=120664) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:53:37.271389 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ad4bb1b6-e3cd-42de-8adf-9df5757184b5 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "POST /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action" status: 202 len: 0 microversion: 2.1 time: 0.131403 Sep 19 16:53:37.272263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1202/2183] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:53:37 2022] POST /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:53:37.278718 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR oslo.messaging._drivers.impl_rabbit [-] [a720f8eb-7716-4853-b50a-5ec1018b2601] AMQP server on 10.222.0.57:5672 is unreachable: Too many heartbeats missed. Trying again in 1 seconds.: amqp.exceptions.ConnectionForced: Too many heartbeats missed Sep 19 16:53:37.279298 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=33,nova_api:INSERT=3 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:37.280226 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=44 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:53:37.281790 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: Server unexpectedly closed connection Sep 19 16:53:37.281966 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:53:37 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b (ip 10.222.0.57) !!! Sep 19 16:53:37.288590 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:37.289356 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:37.299408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:37.299939 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:37.300753 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:37.308016 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:37.308579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:37.309033 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:37.443545 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:37.451531 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e37f840-3d35-4d20-a535-ae749edf5a8d tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.174835 Sep 19 16:53:37.452139 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 982/2184] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:37 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:38.477005 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:38.477716 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:38.484812 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:38.485302 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:38.485740 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:38.486479 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:38.486940 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:38.487362 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:38.711313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:38.719602 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ef5e8841-063d-4291-a71a-8b235a73a96e tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.247841 Sep 19 16:53:38.720160 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1203/2185] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:38 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:53:39.742079 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:39.742737 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:39.750867 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:39.751371 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:39.751816 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:39.753019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:39.753019 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:39.753407 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:39.759906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] [a720f8eb-7716-4853-b50a-5ec1018b2601] Reconnected to AMQP server on 10.222.0.57:5672 via [amqp] client with port 60018. Sep 19 16:53:39.900418 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:39.909887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-68338721-c8c3-43c1-9fd5-1263dbb12295 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.170773 Sep 19 16:53:39.909887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 983/2186] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:39 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:40.936136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:40.937324 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:40.943857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:40.944337 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:40.944774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:40.945468 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:40.945919 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:40.946346 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:41.085178 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:41.093216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58c6fc8a-b213-4f7b-aa49-bea5a863682a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.161179 Sep 19 16:53:41.093798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1204/2187] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:40 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:42.121922 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:42.122542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:42.129411 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:42.129906 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:42.130350 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:42.131088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:42.131542 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:42.131969 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:42.262124 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:42.270232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-72c4ba17-b7ef-4cb7-a89e-97a06ee63f79 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.152235 Sep 19 16:53:42.274997 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 984/2188] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:42 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:43.293383 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:43.293987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:43.300393 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:43.300890 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:43.301325 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:43.302010 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:43.302480 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:43.302902 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:43.444423 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:43.452089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-121a258a-fbb0-4dce-b1d8-ec4137d79f0f tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.162632 Sep 19 16:53:43.453248 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1205/2189] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:43 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:44.475751 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:44.476368 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:44.482811 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:44.483294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:44.484340 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:44.485032 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:44.485492 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:44.485918 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:44.617070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:44.624943 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dd574a4e-f5e7-4857-98e5-7ce7a988580c tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.153113 Sep 19 16:53:44.625509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 985/2190] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:44 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:45.648779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:45.649416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:45.656391 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:45.656882 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:45.657318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:45.658020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:45.658502 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:45.658930 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:45.793438 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:45.801765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-f368ae0a-9a5e-4288-93b7-5a166a0d7caf tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.156980 Sep 19 16:53:45.802320 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1206/2191] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:45 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:46.825547 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:46.826260 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:46.832841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:46.833335 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:46.833768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:46.834488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:46.834958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:46.835362 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:46.987740 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:46.995670 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-382a6e99-d3ab-4cec-b9c8-016e2c922e08 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.174041 Sep 19 16:53:46.996249 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 986/2192] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:46 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:48.019861 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:48.020440 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:48.027157 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:48.027627 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:48.028055 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:48.028737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:48.029183 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:48.029603 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:48.174221 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:48.182265 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ee69cc1b-3957-437b-b2f6-b5edded3bdfd tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.166332 Sep 19 16:53:48.183175 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1207/2193] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:48 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:49.205639 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:49.206245 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:49.212877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:49.213392 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:49.213834 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:49.214563 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:49.215020 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:49.215453 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:49.356488 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:49.364768 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b68d5bd1-58d6-4a18-a4af-4cd553e84072 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1567 microversion: 2.1 time: 0.162943 Sep 19 16:53:49.365344 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 987/2194] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:49 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1567 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:50.390528 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:50.391077 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:50.397685 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.398165 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.398635 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.399342 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.399791 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.400234 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.534263 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:50.541995 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ed8d6a08-23c5-410d-9eee-caa8d974fe45 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1508 microversion: 2.1 time: 0.155387 Sep 19 16:53:50.542589 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1208/2195] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:50 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1508 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:50.564817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:53:50.565416 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:50.571796 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.572280 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.572720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.573417 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.573874 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.574294 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.604265 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120663) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 19 16:53:50.672759 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1d37555e-cd90-4922-b32f-c2b8bd38aed8 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "POST /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action" status: 202 len: 0 microversion: 2.1 time: 0.111525 Sep 19 16:53:50.673391 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 988/2196] 10.222.0.57 () {60 vars in 1278 bytes} [Mon Sep 19 16:53:50 2022] POST /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b/action => generated 0 bytes in 113 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:53:50.682016 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:50.683544 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:50.689403 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.689896 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.690495 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.691250 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:50.691709 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:50.692136 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:50.868898 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:50.878079 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c66d2842-0eb1-4d3b-a978-5b7f7ab1a004 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1516 microversion: 2.1 time: 0.199910 Sep 19 16:53:50.878692 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1209/2197] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:50 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1516 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:51.901641 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:51.902214 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:51.909162 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:51.909663 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:51.910107 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:51.910878 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:51.911390 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:51.912555 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:52.059141 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:52.066962 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ea9512c0-8583-4871-bd4c-d421cff6ffea tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1518 microversion: 2.1 time: 0.169351 Sep 19 16:53:52.067532 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 989/2198] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:51 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1518 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:52.580318 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "name": "network-vif-unplugged", "status": "completed", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:53:52.590631 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:52.591182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:52.592020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:52.603696 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] Creating event network-vif-unplugged:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:53:52.608779 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-e5a6d092-63e2-4f80-a664-6d35dd7a6a7d req-3b1f4261-bdfa-4525-8b05-d2cf62b544b2 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.032698 Sep 19 16:53:52.609648 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1210/2199] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:53:52 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:53:53.091375 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:53.092086 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:53.098512 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:53.098991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:53.099448 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:53.100134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:53.100579 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:53.101028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:53.240645 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:53.248742 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-1f5d0e2d-cc3b-44fe-845d-0e839393ced4 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.161422 Sep 19 16:53:53.249321 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 990/2200] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:53 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:54.272967 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:54.273676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:54.280548 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:54.281112 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:54.281551 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:54.282538 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:54.282994 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:54.283419 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:54.415715 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:54.424348 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-4fd092bd-6494-4ea7-98ad-7a93fa7f4512 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.155976 Sep 19 16:53:54.424926 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1211/2201] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:54 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:55.449063 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:55.449680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:55.456801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:55.457284 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:55.457720 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:55.458444 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:55.458902 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:55.459326 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:55.587259 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:55.595195 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-ed099aca-8d28-4f2e-bb46-4a430e826e91 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.149927 Sep 19 16:53:55.595777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 991/2202] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:55 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:56.619554 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:56.620145 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:56.626530 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:56.627013 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:56.627458 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:56.628140 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:56.628598 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:56.629024 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:56.763825 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:56.772216 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-6a99a5d9-e855-4f13-8624-28fae7cccb55 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.156942 Sep 19 16:53:56.772875 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1212/2203] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:56 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:57.797924 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:57.798577 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:57.806110 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:57.806628 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:57.807070 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:57.807777 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:57.808486 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:57.808913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:57.941887 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:57.949552 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-4e3c67ff-205e-40a9-915b-36bb54987d05 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.156152 Sep 19 16:53:57.950120 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 992/2204] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:57 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:53:58.973848 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:53:58.974482 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:53:58.981189 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:58.981668 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:58.982103 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:58.982826 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:53:58.985636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:53:58.985636 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:53:59.135684 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:53:59.143473 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-731f977d-852c-47d5-8e20-b42bbb373b82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1574 microversion: 2.1 time: 0.173520 Sep 19 16:53:59.144046 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1213/2205] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:53:58 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1574 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:54:00.168817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:54:00.169327 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:54:00.175913 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:54:00.176388 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:54:00.176817 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:54:00.177514 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:54:00.177955 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:54:00.178393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:54:00.317074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:54:00.324889 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-8e1c5f4d-adb7-4820-b976-7889265a3154 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1561 microversion: 2.1 time: 0.160701 Sep 19 16:54:00.325430 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 993/2206] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:54:00 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1561 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:54:02.670998 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "name": "network-vif-plugged", "status": "completed", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:54:02.689504 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:54:02.690105 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:54:02.690719 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:54:02.711194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] Creating event network-vif-plugged:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:54:02.713961 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-9de896e1-02e9-4ef6-91c4-c60bc640ff9e req-f4a06d14-cf16-4b75-9e42-1bc4f91ac200 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.049463 Sep 19 16:54:02.714924 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1214/2207] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:54:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:55:13.410711 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=28,nova_api:UPDATE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:13.411525 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=76,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:13.414311 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=26 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:13.415080 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=79,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:55:13.417393 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:13.418297 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:55:13.426877 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:13.427435 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:13.428841 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:13.429727 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:13.430775 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:13.431207 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:13.463515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Going to try to terminate instance {{(pid=120663) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 19 16:55:13.485228 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:55:13.494917 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:13.495390 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:13.495818 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:13.508203 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] Creating event network-changed:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:55:13.514363 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-7864043d-ba6f-470d-8547-5fa769443fd1 req-be1c8c83-5aee-4f76-8b00-b6878b9b93b3 service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102553 Sep 19 16:55:13.515230 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1215/2208] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:55:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:55:13.534822 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR oslo.messaging._drivers.impl_rabbit [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [18a24371-6ef5-4b8a-bfa4-d68cbb899d17] AMQP server on 10.222.0.57:5672 is unreachable: [Errno 104] Connection reset by peer. Trying again in 1 seconds.: ConnectionResetError: [Errno 104] Connection reset by peer Sep 19 16:55:14.542044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [18a24371-6ef5-4b8a-bfa4-d68cbb899d17] Reconnected to AMQP server on 10.222.0.57:5672 via [amqp] client with port 60284. Sep 19 16:55:14.564436 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-53438f2e-e1c8-4f9e-bf76-627dbf8134c3 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "DELETE /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 204 len: 0 microversion: 2.1 time: 1.155330 Sep 19 16:55:14.565039 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 994/2209] 10.222.0.57 () {58 vars in 1239 bytes} [Mon Sep 19 16:55:13 2022] DELETE /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 0 bytes in 1157 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 19 16:55:14.573987 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:14.574798 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:55:14.581839 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:14.582313 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:14.582797 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:14.583481 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:14.583927 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:14.584347 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:14.720978 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.policy [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120664) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:55:14.728540 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cee58767-dba8-4c9d-af80-30d79dd98f8a tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1449 microversion: 2.1 time: 0.158729 Sep 19 16:55:14.729099 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1216/2210] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:55:14 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1449 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:55:15.752456 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:15.753192 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:55:15.761597 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:15.762076 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:15.762561 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:15.763242 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:15.763673 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:15.764094 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:15.900726 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:55:15.909257 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-b2902f5d-a512-4b13-9b1a-24f52e49ffa6 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1449 microversion: 2.1 time: 0.161198 Sep 19 16:55:15.909832 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 995/2211] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:55:15 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1449 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 19 16:55:16.573737 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4064c43e-48d4-42c9-a243-390cf8f8415b", "name": "network-vif-deleted", "tag": "73215234-3327-4523-bd7c-0b73d1899a74"}]} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:55:16.585295 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:16.585765 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:16.586194 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:16.598344 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.compute.server_external_events [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] Creating event network-vif-deleted:73215234-3327-4523-bd7c-0b73d1899a74 for instance 4064c43e-48d4-42c9-a243-390cf8f8415b on hv-cinder-85837 Sep 19 16:55:16.603676 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [req-21fa3465-5a7e-4fee-a398-7da889c62486 req-226fb95f-157a-4a7d-bef9-877393e1649c service nova] 10.222.0.57 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033531 Sep 19 16:55:16.604286 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1217/2212] 10.222.0.57 () {64 vars in 1317 bytes} [Mon Sep 19 16:55:16 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:55:16.934129 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:16.935402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:55:16.947111 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:16.948017 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:16.949143 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:16.950074 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:16.951150 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:16.952266 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:16.991159 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:16.991646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:16.992078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:16.992534 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.objects.instance [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lazy-loading 'fault' on Instance uuid 4064c43e-48d4-42c9-a243-390cf8f8415b {{(pid=120663) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 19 16:55:17.047646 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.policy [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8a3f7b1c624141feab14c67fa39732a1', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b62d50ba5e3d451dab48a4b6cdc50399', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120663) authorize /opt/stack/nova/nova/policy.py:199}} Sep 19 16:55:17.055764 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d63889d2-5ed7-4367-a392-709ca926bd82 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 200 len: 1224 microversion: 2.1 time: 0.127137 Sep 19 16:55:17.056756 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 996/2213] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:55:16 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 1224 bytes in 129 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 19 16:55:18.081015 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:18.081753 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] [instance: 4064c43e-48d4-42c9-a243-390cf8f8415b] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:55:18.083884 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:55:18.084020 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:55:18 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b (ip 10.222.0.57) !!! Sep 19 16:55:18.095243 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:18.095728 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:18.096171 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:18.096881 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:55:18.097321 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:55:18.097744 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:55:18.112511 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] HTTP exception thrown: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Request method failure captured: Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b HTTP/1.1 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ3uo8d-Uik62NSWM_a5y5eZ2AdYYCHtuJ925CPUBUo9GhgLnp0bX3hXeu8Xbd7INCiBN-qDImEMSZSFsZgRlJraBYT4AJtKeEVGW506vr3MapFuL3mATLGPJhBv8UCBHh6EmsdtG5sj9T921wfcAxNs_hcn7YPhv0vsA6ffoP1wHPI Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: b62d50ba5e3d451dab48a4b6cdc50399 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-TestShelveInstance-318104565 Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:55:18.114291 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b62d50ba5e3d451dab48a4b6cdc50399"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/b62d50ba5e3d451dab48a4b6cdc50399"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/b62d50ba5e3d451dab48a4b6cdc50399"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_b62d50ba5e3d451dab48a4b6cdc50399"}]}] Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-TestShelveInstance-318104565 Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: b62d50ba5e3d451dab48a4b6cdc50399 Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-TestShelveInstance-318104565 Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-TestShelveInstance-318104565-project Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: 8a3f7b1c624141feab14c67fa39732a1 Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-TestShelveInstance-318104565-project Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. Sep 19 16:55:18.115724 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '4064c43e-48d4-42c9-a243-390cf8f8415b'} Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:55:18.116484 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. Sep 19 16:55:18.117642 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:55:18.119006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Returning 404 to user: Instance 4064c43e-48d4-42c9-a243-390cf8f8415b could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:55:18.119006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-5f604368-b1f1-4005-afbd-bbe814e81101 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b" status: 404 len: 111 microversion: 2.1 time: 0.039627 Sep 19 16:55:18.119006 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1218/2214] 10.222.0.57 () {58 vars in 1236 bytes} [Mon Sep 19 16:55:18 2022] GET /compute/v2.1/servers/4064c43e-48d4-42c9-a243-390cf8f8415b => generated 111 bytes in 41 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:55:18.283982 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d4c5a64c-08b7-4bed-99c7-32d76fcce083 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] Calling method '.version_select at 0x7fadc085eb80>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:55:18.309580 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d4c5a64c-08b7-4bed-99c7-32d76fcce083 tempest-TestShelveInstance-318104565 tempest-TestShelveInstance-318104565-project] 10.222.0.57 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1671444004" status: 202 len: 0 microversion: 2.1 time: 0.030993 Sep 19 16:55:18.310354 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 997/2215] 10.222.0.57 () {58 vars in 1254 bytes} [Mon Sep 19 16:55:18 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-1671444004 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:57:01.145068 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=8 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:01.145809 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=12 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:01.147059 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:57:01.147205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Mon Sep 19 16:57:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (ip 10.222.0.57) !!! Sep 19 16:57:01.152408 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:01.152905 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.compute.api [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120664) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:57:01.161743 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:57:01.162225 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:57:01.162691 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:57:01.163378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:57:01.163827 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:57:01.164254 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG oslo_concurrency.lockutils [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120664) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:57:01.177997 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] HTTP exception thrown: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Request method failure captured: Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 HTTP/1.1 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJU81EqY99T609uuwZr9a0sNsoO1V_0zQrwGV9SChkBUTfz8HXeTTUB8w4IvocWqxDJwsnWnMLXY0Ct47KAVDZkhqs20bNAkHi0REaMP2d51-mMiBqEWxa2Sd96fcOxr0jiXyl6uHdarWeygRDHXrca5WbxkjSAyj3ClfovshzB9HSY Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Openstack-Nova-Api-Version: 2.20 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:57:01.179583 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_344bf851ff7d471fba47bde9b6f7e219"}]}] Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: bd190d8e252342ca9232f7a0c966c715 Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: > Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.180782 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': '9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5'} Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:01.181455 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.182491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.183503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Returning 404 to user: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:01.183503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-46e822c0-ba00-4e9f-9ccf-84f27a70dd94 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 404 len: 111 microversion: 2.20 time: 0.037736 Sep 19 16:57:01.183503 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1219/2216] 10.222.0.57 () {60 vars in 1280 bytes} [Mon Sep 19 16:57:01 2022] DELETE /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 19 16:57:01.188524 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:01.189345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=16,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:01.190134 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 19 16:57:01.190267 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Mon Sep 19 16:57:01 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 (ip 10.222.0.57) !!! Sep 19 16:57:01.195638 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Calling method '>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:01.196243 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.compute.api [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] [instance: 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5] Fetching instance by UUID {{(pid=120663) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 19 16:57:01.207725 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:57:01.208232 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:57:01.208666 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:57:01.209413 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Acquiring lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 19 16:57:01.209859 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 19 16:57:01.210278 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG oslo_concurrency.lockutils [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Lock "0c2c7f29-247d-4f27-aaaf-5b73a621e9cb" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120663) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 19 16:57:01.226252 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] HTTP exception thrown: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Request method failure captured: Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 HTTP/1.1 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJU81EqY99T609uuwZr9a0sNsoO1V_0zQrwGV9SChkBUTfz8HXeTTUB8w4IvocWqxDJwsnWnMLXY0Ct47KAVDZkhqs20bNAkHi0REaMP2d51-mMiBqEWxa2Sd96fcOxr0jiXyl6uHdarWeygRDHXrca5WbxkjSAyj3ClfovshzB9HSY Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Openstack-Nova-Api-Version: 2.20 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:57:01.227937 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/344bf851ff7d471fba47bde9b6f7e219"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_344bf851ff7d471fba47bde9b6f7e219"}]}] Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: 344bf851ff7d471fba47bde9b6f7e219 Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-1025235815 Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: bd190d8e252342ca9232f7a0c966c715 Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-AttachVolumeShelveTestJSON-1025235815-project Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: > Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.229078 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5'} Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:01.229689 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. Sep 19 16:57:01.231072 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:01.232237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] Returning 404 to user: Instance 9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:01.232237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d9440a56-dc8a-48c0-b3fa-f11ecbb2e945 tempest-AttachVolumeShelveTestJSON-1025235815 tempest-AttachVolumeShelveTestJSON-1025235815-project] 10.222.0.57 "GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5" status: 404 len: 111 microversion: 2.20 time: 0.043257 Sep 19 16:57:01.232237 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 998/2217] 10.222.0.57 () {60 vars in 1277 bytes} [Mon Sep 19 16:57:01 2022] GET /compute/v2.1/servers/9c25d5d7-a9bf-42f1-bda3-ebd93c1cb9c5 => generated 111 bytes in 44 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 19 16:57:05.783090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ffeee533-40c5-419b-9606-3e6067f3cb3f tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Action: 'create', calling method: .version_select at 0x7fadc31111f0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-550245370"}} {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:57:06.074191 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ffeee533-40c5-419b-9606-3e6067f3cb3f tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 0.423908 Sep 19 16:57:06.075089 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1220/2218] 10.222.0.57 () {60 vars in 1155 bytes} [Mon Sep 19 16:57:05 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 426 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 19 16:57:06.088767 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-3889593e-818b-40ce-b126-786a4e5e2b28 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbfb584c0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:06.270837 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-3889593e-818b-40ce-b126-786a4e5e2b28 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 200 len: 344 microversion: 2.1 time: 0.185905 Sep 19 16:57:06.271401 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 999/2219] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:06 2022] GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 344 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:07.286294 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-bf081803-2a69-4d19-8318-9549aa9c9950 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf63e280>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:07.333358 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-bf081803-2a69-4d19-8318-9549aa9c9950 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 200 len: 345 microversion: 2.1 time: 0.051450 Sep 19 16:57:07.333912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1221/2220] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:07 2022] GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 345 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:07.347610 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-f04e29fd-4682-4d60-bdd6-1ab6450be904 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Action: 'create', calling method: .version_select at 0x7fadbf9eedc0>, body: {"snapshot": {"volume_id": "b0fbe0d7-bb53-485d-a024-68be2f5150d1", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-647217857"}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:57:07.505786 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-f04e29fd-4682-4d60-bdd6-1ab6450be904 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "POST /compute/v2.1/os-snapshots" status: 200 len: 286 microversion: 2.1 time: 0.162652 Sep 19 16:57:07.506331 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1000/2221] 10.222.0.57 () {60 vars in 1162 bytes} [Mon Sep 19 16:57:07 2022] POST /compute/v2.1/os-snapshots => generated 286 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:07.519378 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-68c52547-9d4c-4ccf-a735-7066eb66e8cd tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadc3055e50>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:07.551292 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-68c52547-9d4c-4ccf-a735-7066eb66e8cd tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.036842 Sep 19 16:57:07.551857 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1222/2222] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:07 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:07.563177 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-0b3cb801-81b0-4634-8398-6e588d0ebe07 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbfb1a820>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:07.596515 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-0b3cb801-81b0-4634-8398-6e588d0ebe07 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots" status: 200 len: 289 microversion: 2.1 time: 0.037418 Sep 19 16:57:07.597088 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1001/2223] 10.222.0.57 () {58 vars in 1140 bytes} [Mon Sep 19 16:57:07 2022] GET /compute/v2.1/os-snapshots => generated 289 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:07.608182 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-86262ecf-c0a3-4036-a584-4d8b32d02329 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadc31114c0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:07.638357 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-86262ecf-c0a3-4036-a584-4d8b32d02329 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.033931 Sep 19 16:57:07.638944 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1223/2224] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:07 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:57:08.652279 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-677052a3-32bd-4f56-89fd-538abc04b6a1 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf941a60>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:08.682771 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-677052a3-32bd-4f56-89fd-538abc04b6a1 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.034730 Sep 19 16:57:08.683341 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1002/2225] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:08 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:09.696842 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2aa0523d-6097-44ba-a59c-19749b0ea5b9 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf62a430>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:09.726763 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2aa0523d-6097-44ba-a59c-19749b0ea5b9 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.034117 Sep 19 16:57:09.727331 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1224/2226] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:09 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:10.740203 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-33ce82d3-b91e-4880-b0ff-8665f056f0bb tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf97d310>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:10.770509 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-33ce82d3-b91e-4880-b0ff-8665f056f0bb tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.035148 Sep 19 16:57:10.771090 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1003/2227] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:10 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:11.781416 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_api:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:11.783029 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG dbcounter [-] [120664] Writing DB stats nova_cell1:SELECT=2 {{(pid=120664) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:11.786491 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-966b8877-2744-42c3-b1bc-bed8ed9138c3 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf6a6dc0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:11.819036 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-966b8877-2744-42c3-b1bc-bed8ed9138c3 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 287 microversion: 2.1 time: 0.039086 Sep 19 16:57:11.819708 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1225/2228] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:11 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 287 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:11.829028 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_api:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:11.829958 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG dbcounter [-] [120663] Writing DB stats nova_cell1:SELECT=2 {{(pid=120663) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 19 16:57:11.834596 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-a4e9fd55-e2dc-44fa-a931-03d850f6369d tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbfb57ee0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:11.892774 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-a4e9fd55-e2dc-44fa-a931-03d850f6369d tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "DELETE /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 202 len: 0 microversion: 2.1 time: 0.065152 Sep 19 16:57:11.893345 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1004/2229] 10.222.0.57 () {58 vars in 1254 bytes} [Mon Sep 19 16:57:11 2022] DELETE /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 0 bytes in 67 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:57:11.902172 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-58e32d57-2644-4be3-8aef-261099e051df tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadc3055ee0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:11.933315 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-58e32d57-2644-4be3-8aef-261099e051df tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 200 len: 286 microversion: 2.1 time: 0.035040 Sep 19 16:57:11.933886 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1226/2230] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:11 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 286 bytes in 36 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:12.947571 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2b97620-56be-4b15-8393-08d98a2d1303 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbfac3310>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:12.975201 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-d2b97620-56be-4b15-8393-08d98a2d1303 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] HTTP exception thrown: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2b97620-56be-4b15-8393-08d98a2d1303 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Request method failure captured: Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 HTTP/1.1 Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ9h4P_tffIV08cMRMiCte82c8obckzAC3qEQdVb-oQMXFpRDq5LyWIOH5DKM8eeAorM6q8UDgvkMARLmiLuAIfNzV_vV-RqGzhakGCX6-RyP3LFkBCodIG8uhg0ui340sVp9wHOpauZSbwzlCKjzr3dku90YfZ9kUUk5w54BsiA6rs Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:57:12.978044 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_aefa893439ba4dc48e8a28bda86f9baf"}]}] Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: d211f57d34fc49c79c89131f1b20c38b Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'e380e17f-a77e-4b2c-bc80-bfc348e087e9'} Sep 19 16:57:12.979609 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. (HTTP 404) (Request-ID: req-a76079d1-9007-4ffc-948c-dbd37cd774cd) Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.980743 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:12.981678 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 19 16:57:12.982655 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-d2b97620-56be-4b15-8393-08d98a2d1303 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Returning 404 to user: Snapshot e380e17f-a77e-4b2c-bc80-bfc348e087e9 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:12.983664 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-d2b97620-56be-4b15-8393-08d98a2d1303 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9" status: 404 len: 111 microversion: 2.1 time: 0.036626 Sep 19 16:57:12.984285 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1005/2231] 10.222.0.57 () {58 vars in 1251 bytes} [Mon Sep 19 16:57:12 2022] GET /compute/v2.1/os-snapshots/e380e17f-a77e-4b2c-bc80-bfc348e087e9 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:57:12.988912 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-8791c28c-dad1-43d6-9f01-0a6145d6a762 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf63eaf0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:13.058573 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-8791c28c-dad1-43d6-9f01-0a6145d6a762 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 202 len: 0 microversion: 2.1 time: 0.073893 Sep 19 16:57:13.059558 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1227/2232] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:57:12 2022] DELETE /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 0 bytes in 76 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:57:13.068408 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-2c596ea2-c031-4f53-a969-f72af6ba3805 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbfb57430>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:13.119991 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-2c596ea2-c031-4f53-a969-f72af6ba3805 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 200 len: 344 microversion: 2.1 time: 0.055268 Sep 19 16:57:13.120587 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1006/2233] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:13 2022] GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 344 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:57:14.137037 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c59ae247-468b-4adc-bba4-d09844acddb7 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadc31114c0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:14.168489 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-c59ae247-468b-4adc-bba4-d09844acddb7 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] HTTP exception thrown: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c59ae247-468b-4adc-bba4-d09844acddb7 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Request method failure captured: Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 HTTP/1.1 Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ9h4P_tffIV08cMRMiCte82c8obckzAC3qEQdVb-oQMXFpRDq5LyWIOH5DKM8eeAorM6q8UDgvkMARLmiLuAIfNzV_vV-RqGzhakGCX6-RyP3LFkBCodIG8uhg0ui340sVp9wHOpauZSbwzlCKjzr3dku90YfZ9kUUk5w54BsiA6rs Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:57:14.170472 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_aefa893439ba4dc48e8a28bda86f9baf"}]}] Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: d211f57d34fc49c79c89131f1b20c38b Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'b0fbe0d7-bb53-485d-a024-68be2f5150d1'} Sep 19 16:57:14.172102 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. (HTTP 404) (Request-ID: req-fa06891e-d0c8-42c5-b8ac-285836f5130c) Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.172774 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:14.173712 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 19 16:57:14.174757 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-c59ae247-468b-4adc-bba4-d09844acddb7 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Returning 404 to user: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-c59ae247-468b-4adc-bba4-d09844acddb7 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 404 len: 109 microversion: 2.1 time: 0.041249 Sep 19 16:57:14.175694 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1228/2234] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:14 2022] GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 109 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:57:14.187119 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-53a79cb4-25fe-44ad-9411-59a987535cff tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf9eed30>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:14.219618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-53a79cb4-25fe-44ad-9411-59a987535cff tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] HTTP exception thrown: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-53a79cb4-25fe-44ad-9411-59a987535cff tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Request method failure captured: Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: DELETE /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 HTTP/1.1 Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ9h4P_tffIV08cMRMiCte82c8obckzAC3qEQdVb-oQMXFpRDq5LyWIOH5DKM8eeAorM6q8UDgvkMARLmiLuAIfNzV_vV-RqGzhakGCX6-RyP3LFkBCodIG8uhg0ui340sVp9wHOpauZSbwzlCKjzr3dku90YfZ9kUUk5w54BsiA6rs Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:57:14.221618 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_aefa893439ba4dc48e8a28bda86f9baf"}]}] Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: d211f57d34fc49c79c89131f1b20c38b Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': 'b0fbe0d7-bb53-485d-a024-68be2f5150d1'} Sep 19 16:57:14.222801 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. (HTTP 404) (Request-ID: req-7b8544b4-4d9f-4bb3-be33-db1a97ddda19) Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.223470 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 19 16:57:14.224441 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 19 16:57:14.225347 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-53a79cb4-25fe-44ad-9411-59a987535cff tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Returning 404 to user: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:14.226239 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-53a79cb4-25fe-44ad-9411-59a987535cff tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 404 len: 109 microversion: 2.1 time: 0.040690 Sep 19 16:57:14.230048 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1007/2235] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:57:14 2022] DELETE /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 109 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 19 16:57:14.232252 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae8df410-16c7-4125-ba33-d5872be47a16 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Calling method '.version_select at 0x7fadbf63e820>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:14.260620 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.wsgi [None req-ae8df410-16c7-4125-ba33-d5872be47a16 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] HTTP exception thrown: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae8df410-16c7-4125-ba33-d5872be47a16 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Request method failure captured: Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: request: GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 HTTP/1.1 Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept: application/json Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Accept-Encoding: identity Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Connection: close Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Length: 0 Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Content-Type: application/json Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Host: 10.222.0.57 Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: Openstack-System-Scope: None Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Auth-Token: gAAAAABjKJ9h4P_tffIV08cMRMiCte82c8obckzAC3qEQdVb-oQMXFpRDq5LyWIOH5DKM8eeAorM6q8UDgvkMARLmiLuAIfNzV_vV-RqGzhakGCX6-RyP3LFkBCodIG8uhg0ui340sVp9wHOpauZSbwzlCKjzr3dku90YfZ9kUUk5w54BsiA6rs Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Id: None Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Domain-Name: None Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Identity-Status: Confirmed Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Is-Admin-Project: True Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Id: default Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Domain-Name: Default Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Role: member,reader Sep 19 16:57:14.262711 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Roles: member,reader Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/aefa893439ba4dc48e8a28bda86f9baf"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_aefa893439ba4dc48e8a28bda86f9baf"}]}] Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Id: aefa893439ba4dc48e8a28bda86f9baf Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1082767950 Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Id: default Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Domain-Name: Default Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Id: d211f57d34fc49c79c89131f1b20c38b Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1082767950-project Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: method: Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: exception: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: action_args: {'id': 'b0fbe0d7-bb53-485d-a024-68be2f5150d1'} Sep 19 16:57:14.263804 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. (HTTP 404) (Request-ID: req-49942752-57e3-4bbc-a32a-c4aec8074eb7) Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.264445 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:14.265469 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 19 16:57:14.266725 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: ERROR nova.api.openstack.wsgi Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-ae8df410-16c7-4125-ba33-d5872be47a16 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] Returning 404 to user: Volume b0fbe0d7-bb53-485d-a024-68be2f5150d1 could not be found. {{(pid=120664) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-ae8df410-16c7-4125-ba33-d5872be47a16 tempest-VolumesSnapshotsTestJSON-1082767950 tempest-VolumesSnapshotsTestJSON-1082767950-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1" status: 404 len: 109 microversion: 2.1 time: 0.035979 Sep 19 16:57:14.268092 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1229/2236] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:14 2022] GET /compute/v2.1/os-volumes/b0fbe0d7-bb53-485d-a024-68be2f5150d1 => generated 109 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 19 16:57:15.749171 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-fabf09e6-c9c7-416e-9e07-4e0afa0dcab8 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Action: 'create', calling method: .version_select at 0x7fadbf83cb80>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1670325214", "metadata": {"Type": "work"}}} {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 19 16:57:16.068059 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-fabf09e6-c9c7-416e-9e07-4e0afa0dcab8 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 0.438202 Sep 19 16:57:16.068680 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1008/2237] 10.222.0.57 () {60 vars in 1156 bytes} [Mon Sep 19 16:57:15 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 440 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 19 16:57:16.083090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-2e193a1f-f6ca-4a1d-9193-40e965c21014 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadbf63e160>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:16.142205 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-2e193a1f-f6ca-4a1d-9193-40e965c21014 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 200 len: 353 microversion: 2.1 time: 0.063553 Sep 19 16:57:16.143090 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1230/2238] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:16 2022] GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 353 bytes in 66 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:17.162336 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-9e079fd5-dd41-40a9-96f0-9216db6e3343 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadbfb57ee0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:17.225324 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-9e079fd5-dd41-40a9-96f0-9216db6e3343 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 200 len: 354 microversion: 2.1 time: 0.067477 Sep 19 16:57:17.225895 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1009/2239] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:17 2022] GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 354 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 19 16:57:17.242638 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-cb83af52-179b-4cb3-982e-ba75d24e9cbc tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadc317a550>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:17.298009 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-cb83af52-179b-4cb3-982e-ba75d24e9cbc tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 200 len: 354 microversion: 2.1 time: 0.061223 Sep 19 16:57:17.298617 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1231/2240] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:17 2022] GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 354 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:17.321402 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-810f5c31-41cd-4f2d-abd6-e3a6625b326b tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadbfb57700>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:17.393479 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-810f5c31-41cd-4f2d-abd6-e3a6625b326b tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "DELETE /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 202 len: 0 microversion: 2.1 time: 0.076081 Sep 19 16:57:17.394217 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1010/2241] 10.222.0.57 () {58 vars in 1248 bytes} [Mon Sep 19 16:57:17 2022] DELETE /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 0 bytes in 78 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 19 16:57:17.403319 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: DEBUG nova.api.openstack.wsgi [None req-158ae9db-d601-4e49-b733-653e7f804b60 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadc3119dc0>' {{(pid=120664) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:17.462675 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: INFO nova.api.openstack.requestlog [None req-158ae9db-d601-4e49-b733-653e7f804b60 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 200 len: 353 microversion: 2.1 time: 0.063090 Sep 19 16:57:17.463280 dv-cinder-858370-2-iscsi devstack@n-api.service[120664]: [pid: 120664|app: 0|req: 1232/2242] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:17 2022] GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 353 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 19 16:57:18.477674 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd042c47-301d-446e-982d-f11bb903e2e1 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Calling method '.version_select at 0x7fadbfac3af0>' {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 19 16:57:18.507172 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.wsgi [None req-dd042c47-301d-446e-982d-f11bb903e2e1 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] HTTP exception thrown: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd042c47-301d-446e-982d-f11bb903e2e1 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Request method failure captured: Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: request: GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 HTTP/1.1 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept: application/json Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Accept-Encoding: identity Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Connection: close Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Length: 0 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Content-Type: application/json Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Host: 10.222.0.57 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: Openstack-System-Scope: None Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: User-Agent: python-urllib3/1.26.12 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Auth-Token: gAAAAABjKJ9rYC8cCCuPficDNv5GAjGmJQN1lsltNM3UPxtKj87-KtiNay7NPNYmF32w04iUaPL25hWtcjrZGIQ2P3qXnMHigb2lpL1tK2RSMTMwUDoU8FXB8Ft4C9FqIBcrBUn6i1ZWy8g5wflmTbHmVHpjRAZc9yPusSsN6MwaaCm3P30-HvA Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Id: None Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Domain-Name: None Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Identity-Status: Confirmed Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Is-Admin-Project: True Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Id: default Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Domain-Name: Default Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Id: f1a09fac0d9344fc84e0d7d23a252c82 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Project-Name: tempest-VolumesGetTestJSON-464520317 Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Role: member,reader Sep 19 16:57:18.509156 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Roles: member,reader Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Service-Catalog: [{"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/placement"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/f1a09fac0d9344fc84e0d7d23a252c82"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2/f1a09fac0d9344fc84e0d7d23a252c82"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57:9696/networking"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/compute/v2.1"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/identity"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.57/volume/v3/f1a09fac0d9344fc84e0d7d23a252c82"}]}, {"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "adminURL": "http://10.222.0.57:8080", "publicURL": "http://10.222.0.57:8080/v1/AUTH_f1a09fac0d9344fc84e0d7d23a252c82"}]}] Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant: tempest-VolumesGetTestJSON-464520317 Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Id: f1a09fac0d9344fc84e0d7d23a252c82 Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-Tenant-Name: tempest-VolumesGetTestJSON-464520317 Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User: tempest-VolumesGetTestJSON-464520317-project Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Id: default Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Domain-Name: Default Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Id: 06217a545f714b509145f0ce4dd62ad9 Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: X-User-Name: tempest-VolumesGetTestJSON-464520317-project Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: method: Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: exception: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: action_args: {'id': '4fa0eccd-5c19-45c7-a3e1-651229c98244'} Sep 19 16:57:18.510568 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: {{(pid=120663) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. (HTTP 404) (Request-ID: req-ebca156b-b8ce-4663-952e-6f0eebd36fc9) Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:18.511842 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 19 16:57:18.512819 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 19 16:57:18.513717 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: ERROR nova.api.openstack.wsgi Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: DEBUG nova.api.openstack.wsgi [None req-dd042c47-301d-446e-982d-f11bb903e2e1 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] Returning 404 to user: Volume 4fa0eccd-5c19-45c7-a3e1-651229c98244 could not be found. {{(pid=120663) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: INFO nova.api.openstack.requestlog [None req-dd042c47-301d-446e-982d-f11bb903e2e1 tempest-VolumesGetTestJSON-464520317 tempest-VolumesGetTestJSON-464520317-project] 10.222.0.57 "GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244" status: 404 len: 109 microversion: 2.1 time: 0.037309 Sep 19 16:57:18.515092 dv-cinder-858370-2-iscsi devstack@n-api.service[120663]: [pid: 120663|app: 0|req: 1011/2243] 10.222.0.57 () {58 vars in 1245 bytes} [Mon Sep 19 16:57:18 2022] GET /compute/v2.1/os-volumes/4fa0eccd-5c19-45c7-a3e1-651229c98244 => generated 109 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)